Warren Buffett is known as one of the best investors of all time, and for good reason. In roughly 60 years as CEO of Berkshire Hathaway (BRKA +0.66%)(BRKB +0.69%), Buffett generated a staggering 6,099,294% return for shareholders. That’s not a typo. Berkshire’s returns under Buffett’s leadership were roughly 132 times what an investor would have gained in the S&P 500 in the same period.

However, Buffett’s stellar performance didn’t happen in a straight line. In fact, of those 60 years, Berkshire produced a negative return in 11 of them, including a decline of nearly 49% in a single year.

Berkshire’s stellar performance was not driven by how Buffett navigated bull markets. It was driven in large part by Warren Buffett’s ability to identify and capitalize on market opportunities when the market was in a bad state.

Only five years to get excited about?

In a recent CNBC interview, Buffett reflected on his tenure at Berkshire by saying, “We’ve been — up to 60 years I’ve been in the business. Probably five of them really juicy.”

Looking at his track record, although Berkshire gained in more years than it declined, there were some clear standouts, such as a 130% gain in 1976 and a 103% gain in 1979. But that’s not what he was referring to.

Buffett’s most exciting years weren’t when Berkshire’s stock price went up the most. In fact, Buffett is notorious for not really caring about what Berkshire’s stock does over any short period.

The five years Buffett likely considers the “juiciest” are when nobody thought it was a good time to buy stocks. The most opportunistic years. He didn’t specify which ones, but he’s likely referring to years like the 2008 financial crisis, when Berkshire’s financial flexibility allowed Buffett to scoop up shares of Goldman Sachs (GS +4.96%) cheaply. This period also led to the savvy acquisition of BNSF Railroad, which is now one of Berkshire’s most valuable businesses.

Two that are almost certainly on Buffett’s list are 1973 and 1974, when the S&P 500 lost roughly half of its value due to an oil embargo, stagflation, and other issues. The market offered many opportunities for long-term investors in these years, and Buffett has referred to his 1973 investment in Washington Post shares as one of the defining investments of his career.

Another example was 1991, when the Savings and Loan collapse and a recession caused financial stocks to drop sharply. It was at this point that Buffett started buying shares of American Express (AXP +3.98%), which remains one of Berkshire’s core stock positions today.

Don’t be afraid of the juicy years

As an investor who had just started a few years before the financial crisis, I can say firsthand that a 50%+ drawdown in the S&P 500 can be scary. But in retrospect, it was also the period in my investing career when the market was most full of opportunities.

Buffett once said, “Opportunities come infrequently. When it rains gold, put out the bucket, not the thimble.” In other words, when opportunities arrive, don’t let your natural fear of the market’s volatility cause you to tiptoe in. When one of the juicy years presents itself, keep a long-term mentality and buy top-quality businesses on sale.

Of course, Buffett didn’t put money to work only in those years. He bought and sold stocks regularly throughout his career, regardless of what the economy or overall stock market was doing. But many of the moves that produced game-changing returns in Berkshire’s portfolio were made when the stock market was a scary place to be.