Hey, does anyone want to help me with a search engine?

Its going to search trough https://microstudio.dev/community/

Its gonne work like this

A crawler is going to go to the most recent community post and is gonne get the text inside (not the comments) when it has done that with all the posts it will save those

After that you can search someting then it looks at the post that match the search you made. the most and goes to that one and looks at the comments and will give you those (or i possible a simple version of the comment)

A search engine is unfortunately a bit more work than that.

The crawler is only a piece of a larger puzzle.

Through previous work, I somewhat analyzed what a search engines is made up of.

Of course you need the crawler, and the interface of the actual engine. But you also need a so called Indexer, Ranker and Query Engine.

The indexer extracts all important information from what the crawlers gathered, and the ranker "ranks" the websites to certain terms. So that if I look up "Collision", the most relevant posts are shown up at top.

And the query engine just effectively searches for all the data. Using a database like mysql or mariadb is recommended for storing all the data. If you don't wanna set it up, there might be possible to utilize firebase in some way. That's beyond me understanding tho.

Look up what tokens are and what a Markov chain is (Those are both essential to make this work)