Larry Page and Sergey Brin decide to copy others and create a search engine. The idea for search wasn't new, Jerry Yang and David Filo (also from Stanford) had created "Yahoo!" (Yet Another Hierarchically Organized Oracle) to do the same thing, years earlier (as had others like World Wide Web Worm or Excite ). But while Yahoo was better organized way to find things by topics (using human curation) -- and it had a fallback index pile of everything else, Google's Page Rank algorithm, of ranking things by how many others pointed to it, just worked a little bit better. Google's idea was that humans are expensive and subjective, so replace them with algorithms that can encode their biases consistently.
There was also AltaVista which was a rival and doing better than Yahoo and Google, however, it was acquired by Compaq, and like they did with everything else, they fucked it up until everyone else surpassed it, and it became irrelevant. Compaq was the Novell of the 1990's -- everything they touched they destroyed. Google investors owe them billions. If your Résumé has Compaq or Novell in your background, and you were a Sr. Exec that had authority over the decision, you show you had the Scatalogical touch. (A less appealing version of the Midas touch).
NOTE: Google has never been good at search: as in users always get a pile of crap they don't want. And often can't find what they do want. But Google is slightly less inept than the alternatives. Proving you don't have to be good, just better. That's not meant as a barb or a bias. Look at the failures to find what you want, or how many iterations to find what you look for. Some of that is because you are flawed in what you ask for -- but much of it is because Google doesn't know how to ask clarifying questions, or group responses, or now how to narrow down, and what to exclude. If you have a simple search, it's not horrible. If you want something that's a commonly overloaded term (that can mean more than one thing), it's a shitshow of distractions and guessing games, trying to exclude stuff you don't want.
While algorithms offered inferior results, it did scale better that an index and it seemed quicker. Instead of 10 clicks and loads to get what you wanted you typed something, it gave you results, then you refined your search 3 or 4 more times, to get what you actually wanted, or gave up -- but everything was quickly wrong, instead of slowly right. With a few thousand different websites, Yahoo could be well organized. With a few hundred million, it could not. It was a lot easier to type in a word and get a result, than tunnel-click through deeper and deeper subtopic hierarchy to find to the variants you wanted -- so search won over organization, because 70% of the time, it was "good enough".
Yahoo also became distracted by the "portal" aspects of their site, curation, customization of the landing page with everything people wanted to add on a home page (like weather, stock, news, favorite links, and so on), this slowed their process with distractions (and page slowing ads and annoyances). Google catered to just search, fast, simple (low bandwidth) interface, and was ad-less (early on), and avoided all that messy graphics and formatting stuff that slowed pages down. It was mediocre at one thing: search results quickly. But that was better than being poor at many things: you don't have to be faster than the bear, just your buddy. And they are better than the rest.
Since everyone needed search, they almost had an instant monopoly. And they were smart enough to listen to people screaming at them, that eventually they had to make money. So they acquired anyone doing Internet Advertising, and created a revenue stream. They became like newspapers or other media: editorial gets the eyeballs, you make money off the advertising. Only their editorial was a few shell scripts that that answered people's questions with mediocre results. The worse it worked, the more time people would spend on site trying to find the results they needed. (Assuming they were still better than nothing or the competition).