40 lines
2.8 KiB
Markdown
40 lines
2.8 KiB
Markdown
# TODO
|
|
|
|
- [x] chore: initial commit
|
|
- [x] Deploy first staging version (v1.0.0-staging.1)
|
|
- [x] Wikipedia Database Dump
|
|
- [x] Download SQL files
|
|
- [x] Extract SQL files
|
|
- [x] Tables structure `CREATE TABLE`
|
|
- [x] `page.sql` (`pages` tables)
|
|
- [x] `pagelinks.sql` (`internal_links` tables)
|
|
- [x] Adapt downloaded SQL files
|
|
- [x] `page.sql` (`pages` tables)
|
|
- [x] `pagelinks.sql` (`internal_links` tables)
|
|
- [x] Import SQL files
|
|
- [ ] Try `SELECT count(*) FROM internal_links il WHERE il.source_id = (SELECT p.id FROM pages p WHERE p.title = 'Linux'); -- Count of internal links for 'Linux' page`
|
|
- [ ] Move from POC (Proof of concept) in `data` folder to `apps/cli` folder
|
|
- [ ] Documentation how to use + Last execution date
|
|
- [ ] Rewrite bash script to download and extract SQL files from Wikipedia Database Dump to Node.js for better cross-platform support and easier maintenance + automation, preferably one Node.js script to generate everything to create the database
|
|
- [ ] Verify file content up to before inserts, to check if it matches last version, and diff with last version
|
|
- [ ] Update logic to create custom `internal_links` table to make it work with latest wikipedia dumps (notably concerning the change in `pagelinks.sql` where the title is not included anymore, but instead it uses `pl_target_id`, foreign key to `linktarget`), last tested dumb working `20240420`
|
|
- [ ] Handle redirects
|
|
- [ ] Implement REST API (`api`) with JSON responses ([AdonisJS](https://adonisjs.com/)) to get shortest paths between 2 pages
|
|
- [ ] Implement Wikipedia Game Solver (`website`) with inputs, button to submit, and list all pages to go from one to another, or none if it is not possible
|
|
- [ ] Implement toast notifications for errors, warnings, and success messages
|
|
- [ ] Implement CLI (`cli`)
|
|
- [ ] Add docs to add locale/edit translations, create component, install a dependency in a package, create a new package, technology used, architecture, links where it's deployed, how to use/install for end users, how to update dependencies with `npx taze -l` etc.
|
|
- [ ] GitHub Mirror
|
|
- [ ] Delete `TODO.md` file and instead use issue for the remaining tasks
|
|
|
|
## Links
|
|
|
|
- <https://www.sixdegreesofwikipedia.com/> and <https://github.com/jwngr/sdow>
|
|
- <https://github.com/shyamupa/wikidump_preprocessing>
|
|
- <https://www.mediawiki.org/wiki/API:Allpages>
|
|
- <https://www.thewikigame.com/>
|
|
- How to get all URLs in a Wikipedia page: <https://stackoverflow.com/questions/14882571/how-to-get-all-urls-in-a-wikipedia-page>
|
|
- <https://en.wikipedia.org/w/api.php?action=query&titles=Title&prop=links&pllimit=max&format=json>
|
|
- [YouTube (Amixem) - WIKIPEDIA CHALLENGE ! (ce jeu est génial)](https://www.youtube.com/watch?v=wgKlFNGU174)
|
|
- [YouTube (adumb) - I Made a Graph of Wikipedia... This Is What I Found](https://www.youtube.com/watch?v=JheGL6uSF-4)
|