This repository has been archived on 2024-10-12. You can view files and clone it, but cannot push or open issues or pull requests.
wikipedia-game-solver/TODO.md

66 lines
4.2 KiB
Markdown
Raw Normal View History

2024-07-24 12:35:33 +02:00
# TODO
- [x] chore: initial commit
2024-07-24 12:35:33 +02:00
- [x] Deploy first staging version (v1.0.0-staging.1)
- [x] Wikipedia Database Dump
- [x] Download SQL files
- [x] Extract SQL files
- [x] Tables structure `CREATE TABLE`
2024-08-08 03:21:53 +02:00
- [x] `page.sql` (`pages` table)
- [x] `pagelinks.sql` (`internal_links` table)
- [x] Adapt downloaded SQL files
2024-08-08 03:21:53 +02:00
- [x] `page.sql` (`pages` table)
- [x] `pagelinks.sql` (`internal_links` table)
- [x] Import SQL files
- [x] Try `SELECT count(*) FROM internal_links il WHERE il.from_page_id = (SELECT p.id FROM pages p WHERE p.title = 'Linux'); -- Count of internal links for 'Linux' page`
- [x] Try:
```sql
SELECT il.to_page_id, pl.title
FROM internal_links il
JOIN pages pl ON pl.id = il.to_page_id
WHERE il.from_page_id = (
SELECT p.id FROM pages p WHERE p.title = 'Node.js'
);
```
- [ ] Move from POC (Proof of concept) in `data` folder to `apps/cli` folder
- [ ] Documentation how to use + Last execution date
- [ ] Rewrite bash script to download and extract SQL files from Wikipedia Database Dump to Node.js for better cross-platform support and easier maintenance + automation, preferably one Node.js script to generate everything to create the database
- [ ] Verify file content up to before inserts, to check if it matches last version, and diff with last version
- [ ] Update logic to create custom `internal_links` table to make it work with latest wikipedia dumps (notably concerning the change in `pagelinks.sql` where the title is not included anymore, but instead it uses `pl_target_id`, foreign key to `linktarget`), last tested dumb working `20240420`
- [ ] Handle redirects
- [ ] Implement REST API (`api`) with JSON responses ([AdonisJS](https://adonisjs.com/)) to get shortest paths between 2 pages
2024-08-10 02:32:34 +02:00
- [x] Init AdonisJS project
- [x] Create Lucid models and migrations for Wikipedia Database Dump: `pages` and `internal_links` tables
- [x] Implement `GET /wikipedia/pages?title=Node.js` to search a page by title (not necessarily with the title sanitized, search with input by user to check if page exists)
2024-08-10 02:32:34 +02:00
- [ ] Implement `GET /wikipedia/pages/internal-links/paths?from=Node.js&to=Linux` to get all the possible paths between 2 pages with titles sanitized
2024-08-12 14:21:34 +02:00
- [x] Setup tests with database + add coverage
- [ ] Setup HTTP Requests logging in development (not needed in `test` mode)
- [ ] Setup Health checks
- [ ] Setup Rate limiting
- [ ] Share VineJS validators between `website` and `api`
2024-08-10 02:32:34 +02:00
- [ ] Implement Wikipedia Game Solver (`website`)
- [x] Init Next.js project
- [ ] Try to use <https://www.npmjs.com/package/@tuyau/client> for API calls
- [ ] Hard code 2 pages to test if it works with `console.log` in the browser
- [ ] Implement a form with inputs, button to submit, and list all pages to go from one to another, or none if it is not possible
- [ ] Add images, links to the pages + good UI/UX
- [ ] Autocompletion page titles
- [ ] Implement toast notifications for errors, warnings, and success messages
2024-07-24 12:35:33 +02:00
- [ ] Implement CLI (`cli`)
2024-08-10 02:32:34 +02:00
- [ ] Init Clipanion project
- [ ] Implement `wikipedia-game-solver internal-links --from="Node.js" --to="Linux"` command to get all the possible paths between 2 pages.
- [ ] Add docs to add locale/edit translations, create component, install a dependency in a package, create a new package, technology used, architecture, links where it's deployed, how to use/install for end users, how to update dependencies with `npx taze -l` etc.
- [ ] GitHub Mirror
2024-08-08 03:21:53 +02:00
- [ ] Delete `TODO.md` file and instead use issues for the remaining tasks
2024-07-24 12:35:33 +02:00
## Links
2024-07-30 09:10:02 +02:00
- <https://www.sixdegreesofwikipedia.com/> and <https://github.com/jwngr/sdow>
2024-07-28 17:57:41 +02:00
- <https://github.com/shyamupa/wikidump_preprocessing>
- <https://www.mediawiki.org/wiki/API:Allpages>
2024-07-24 12:35:33 +02:00
- <https://www.thewikigame.com/>
- How to get all URLs in a Wikipedia page: <https://stackoverflow.com/questions/14882571/how-to-get-all-urls-in-a-wikipedia-page>
- <https://en.wikipedia.org/w/api.php?action=query&titles=Title&prop=links&pllimit=max&format=json>
- [YouTube (Amixem) - WIKIPEDIA CHALLENGE ! (ce jeu est génial)](https://www.youtube.com/watch?v=wgKlFNGU174)
- [YouTube (adumb) - I Made a Graph of Wikipedia... This Is What I Found](https://www.youtube.com/watch?v=JheGL6uSF-4)