What is the government's budget deficit? It is the difference between the government's expenditures and its income in a given year. The government's expenditures must be financed--if not by income, then by borrowing. Hence, the budget deficit is the contribution that a given year makes to the public debt.
What causes the budget deficit to grow? If the government's expenditures rise, other things being equal, or if the government's income falls, other things being equal, the budget deficit increases. That's it. It is always one, the other, or both that cause the deficit to swell. Consequently, any recipe for shrinking the budget deficit must involve less expenditures, more income, or some mix of the two.
Why does the budget deficit matter? The government's savings is, like for a household, the difference between its income and its expenditures in a given year. Thus, a bigger budget deficit implies less public savings. As long as decreases in public savings do not induce equal or offsetting increases in private savings, therefore, a bigger budget deficit makes for less national savings. Less national savings either contracts the supply of loanable funds, expands the demand for loanable funds, or both, raising the natural rate of interest. A higher natural rate of interest, other things being equal, stimulates the aggregate demand for goods and services (AD).
The central bank is usually tasked with managing AD by way of managing the market rate of interest. If AD was previously on target, by the central bank's lights, then the market rate of interest will be raised in order to offset the AD stimulus induced by the swelling of the budget deficit. Thus, under normal circumstances, a bigger budget deficit raises interest rates. At higher interest rates, private borrowers do not wish to borrow as much, which means that the government's additional borrowing is "crowding out" some private borrowing. In most cases, therefore, it only makes sense to increase the budget deficit if we have reason to believe that the government's borrowing will prove to be more productive than the private investment it is crowding out.
Additionally, the government's financing need not come solely from domestic lenders. Foreign lenders who wish to buy the government's debt will first have to exchange their own currency for the domestic currency. This expands the demand for the domestic currency, causing it to strengthen in foreign exchange markets. A stronger currency makes importing from the domestic country more expensive, causing the domestic country's exports to fall. Thus, a bigger budget deficit also produces a bigger trade deficit.
There is also a question of how the debt produced by bigger budget deficits is to be retired. One option is to roll over the debt, which is to say, borrow some more to make principal and interest payments on the debt in a timely manner. This is feasible so long as interest rates on government debt remain low. Another option is to increase the government's savings, by increasing its income (collecting more tax revenues), decreasing its expenditures, or a bit of both, in order to pay down the debt. A final option is to default on the debt.
The first option is the most painless. A government's ability to service its debt depends upon its power to tax its citizens, for this is its only (significant) source of income. As a result, the higher a country's GDP, the more able it is to service its debt, for this makes possible the raising of greater tax revenues. A government's ability to service its debt, therefore, is best revealed by its debt-to-GDP ratio, rather than its debt in absolute terms. So long as the country's economy is growing as fast, or faster, than the government's debt, investors do not have cause for concern. In such cases, the government typically enjoys low interest rates, making rolling over its debt feasible (forever, potentially).
The second option is more or less painful depending on the circumstances. Economic growth raises the government's income--often reducing its expenditures on safety net programs, too--thereby increasing its savings without anyone making sacrifices. In countries with serious growth problems, or serious debt problems, though, this may not be enough. In such cases, sacrifices must be made. Either tax rates go way up, expenditures go way down, or both. The more growth-friendly the reforms, and the more slowly they're phased in, the less painful the transition. The longer a country waits to make these sacrifices, the more hurried and more extreme they have to be.
If a country's debt problems get out of hand, the result is a 'hard landing'. Investors lose confidence in the government's ability or willingness to service its debts, so they charge higher interest rates for the greater risk they're bearing. Higher interest rates, in turn, worsen the government's finances, causing investors to charge even higher interest rates, worsening the problem further, etc. Eventually, it becomes impossible for the government to service its debts through either economic growth or budget reforms (see, e.g., Greece). At that point, the choice is between explicit default, or implicit default through debt monetization. Only countries with their own currencies have the latter option, and this option usually serves to slow the aforementioned interest rate death spiral. Both courses subject economies to severe financial/macroeconomic dislocation in the short run (due to either banking panics or currency crises), and much higher borrowing costs in the long run.
There's more to say on the subject, but the short story is: maintain budget deficits when crowding out is not much of a problem (i.e., when financing productive public investments, in the midst of a recession, etc.), but maintain budget surpluses the rest of the time, so that investors chillax and the government doesn't risk a hard landing. Don't wait to fix long-term deficit problems--the more thoughtfully they're approached, and the more slowly they're implemented, the happier the outcome. And don't worry so much about future generations. Debt is indeed a burden upon them, but they'll be richer than us, so, whatever.
ADDENDUM: In the United States, bigger budget deficits have not, of late, raised interest rates. The reason is that the US central bank, the Federal Reserve, would like to see lower interest rates, but its policy rate is near zero (its lower bound). As a result, the Fed has not raised interest rates in order to offset bigger budget deficits--in effect, it is having fiscal policy to do some of the work (of stimulating AD) that would normally be performed entirely by monetary policy. Nor have investors lost confidence in the US government (yet!), because they expect currently bloated budget deficits to shrink substantially as the US economy returns to full employment. The real deficit problem in the US stems from the implicit liabilities of Medicare, Medicaid, Social Security, etc. Without reform, these programs imply (under plausible assumptions) unsustainably large budget deficits that would, almost surely, shake investor confidence. Investors have been giving us quite a bit of breathing room to reform these programs, but the longer we take to do so, the more likely it is that we (or the next generation) will suffer a hard landing.