Why was this border agent rude to me?

I use a very similar passport case, and like you I keep one or two cards in the back of it - although not money (in my case these cards are only cards that I may need at immigration, such as my US Green Card or my APEC Business Travel Card).

Passport cases, even thin plastic ones, often interfere with the readers that immigration staff use to scan the biographical page on the passport. Many countries I've visited have signage in the immigration area instructing you to remove your passport from its cover before handing it to immigration staff. On more than one occasion when I've failed to do this I've had the officer hand the passport back to me and ask me to remove the case.

Having additional items in the passport cover also leads to the potential that they may be dropped by the agent as they try and remove the passport from the case - in effect your actions are making the officers job more difficult.

Then we come to the money. In general, handing money to an immigration agent is a clear-cut sign that you are attempting to bribe them. This would be the case whether the money was handed to them separately, or within the passport. Having the money "hidden" in the passport doesn't change that fact, it just makes it appear more like you are trying to make the act of bribing the official less obvious.

Once the officer is of the belief that you attempted to bribe them, they will naturally become more suspicious and attempt to determine why you felt the need to bribe them - such as the fact that you may have been using a forged passport.

It doesn't matter whether your intent was to actually bribe the official or not - the fact is that you acted in the exact way that someone that was trying to bribe them would act, so they have to presume that was your intent. I would say that you were very lucky if the only additional action they took was to view your passport with a magnifying glass - it could have ended much worse for you!

There are lots of good answers here, but just to add - I would strongly advise against keeping money in your passport. You actually were lucky this time. In some countries, in particular, in Eastern Europe, where the situation with corruption is pretty bad still, the border control office might charge you with an attempt to give a bribe. They even have reminders at the border sometimes to make sure there is no money in your passport before you hand it over.

You don't want to start your trip with a criminal charge, so better find a different place to keep your money.

I think the other answers are good, but there's also a possibility that the border agent was being brusque with you to see how you would respond, i.e., s/he was doing behavioral profiling. For instance, if an authority figure asks a pointed question to an innocent person, that might just provoke confusion. But if s/he asks the question of a guilty person, then the person might become visibly nervous and over-explain himself. (That is the theory anyway.) I've had innumerable encounters at borders and airport where the border agent's questions and demeanor made absolutely no sense, aside from its being part of behavioral profiling.

I also had a memorable experience of inadvertently saying something funny to a Canadian border agent, who then had a difficult time maintaining his gruff persona for the rest of the interaction.

Travelling to another country can be a fun and exciting experience. However, don't always expect a friendly experience at the border, especially if you are visiting outside the United States, Canada or United Kingdom where people are unusually polite.

Border officers have long and difficult days trying to sift out liars and standard entries all day. It can be frustrating to not speak your native language all day and work with people who don't understand you or anything, or are blatantly breaking immigration law.

As soon as you said, you keep $100 in your passport, I am instantly reminded that many will try to bribe the border guard to let them in. And in many countries where corruption is rampant, guards can lose their jobs for not handling this according to procedure.

Chances are you set off a few red flags and the border guard was required to do some additional checks. We really won't ever know, though.

Also remember, what is rude to you might be just a typical work culture behaviour.