I am inclined, flippantly, to say, "But they are here now. Look at all the unexplained UFO reports."
In fact, the history suggests that the Fermi Paradox grew out of a discussion about UFOs, as Fermi and others walked to lunch. According to that history, they were talking about the latest UFO reports. Fermi then said that if there were a multitude of civilizations in our galaxy, it was strange that no evidence of them had been found.
So, how do we answer the question?
We can always look at the assumptions made. First, that there are a multitude of civilizations out there. Maybe there are but a few scattered throughout the galaxy which would mean they are probably separated by tens of thousands of light years. Contact among them would be sparse until one or more conquered the problem of interstellar distances.
Maybe we have found no radio trace (or limited radio traces) because they have not reached that level of technology... or more likely, have reached it and moved beyond it. We search for alien radio signals based on some human assumptions given the nature of the radio sources in the sky, but other creatures on other planets might not use those same assumptions. We might be searching in vain because we think like humans and not aliens.
Carl Sagan has postulated that we could expect visitation by an extraterrestrial civilization about once every ten thousand years... though I don’t know how he came up with that number. But let’s say it’s accurate. That would mean that sometime in the last ten thousand years aliens arrived on Earth, and that would mean that they would have found our civilizations.
It really doesn’t matter when they arrived or the state of the civilizations they found. I would think that once you found something like that, you’d be inclined to keep watch on it, if for no other reason than it is another intelligent race. And if that is true, then the number of visits would increase as we advanced. Once we reached an industrial civilization, once we began developing machines to make our lives easier, rockets that could leave the planet, atomic power, and began radiating electromagnetic signals that would make us brighter than almost anything else in the Solar System in that spectrum, they would come by to take a look. They would visit with more frequently...
And isn’t that the situation we have today? Reports of UFOs growing from the beginnings of the industrial revolution until we have the thousands of good, solid cases.
Don’t we have some good physical trace cases including radar/visual sightings? Aren’t there some good photographs that can only be explained as either alien spaceships or hoaxes with no middle ground? Aren’t there some very puzzling sightings that involve multiple witnesses, instrumentality and other evidence?
So, the answer to the Fermi Paradox might not seem so flippant when we look at all the evidence. Maybe the answer is that we have been visited but we have failed to recognize the visitors. We’re so busy arguing about the reality of UFO sightings that we have ignored the bigger questions which is who are they and why are they here.