Compare commits

..

10 Commits

3 changed files with 85 additions and 37 deletions

View File

@ -4,6 +4,17 @@ Zǎoshang hǎo zhōngguó xiànzài wǒ yǒu BING CHILLING 🥶🍦 wǒ hěn xǐ
This project is a meme but it works until you have too many favorites to scrape and you get rate limited, or so I was told by a friend, not that I would know.
This is the culmination of 3 years of computer science at an expensive university and totally what i should be doing in my free time
Now has code that will skip a query to their API if the torrent file for that ID already exists, this may be necessary as scraping continously will get you rate limited after a while. However you seem to be allowed to download thousands of files at a time.
This repo can be used to basically download all of your favorites. It uses a method that is fairly quick, as it will dump the torrents linked to all of the mangas you saved in your favorites. This is a 3 step process that involves first dumping all the ids of the mangas you have fav'd then download all of the torrent files and then download the actual mangas using your favorite torrent client. you can do that the 100% automatic way if you use qbittorent as its stated below.
## why use this ?
Other downloaders I found seem to be broken and mainly targeted for single download use-case as they seem to entirely scrape the manga from the website meaning that the tools that I found (or the ones that work similarly to this that are broken) would have visited the manga webpage and tried to scrape the single image assets from the website and then attempted to rebuild the manga locally on your pc. I think that aproach is not sustainable for more than one download use case.
I use the torrent file aproach, by simply supplying the id of your favorite mangas this tool can query nhentai api and download the official torrent file. I think my aproach works best than others because a torrent client like qbittorent can manage 10k downloads seamlessly and more. My aproach is tailored to massive backup effort of all of your favorites in an automatic manner. once all the files are added to the qbittorent client you can leave it on and let it download everything for you. It's the perfect solution to a problem that many people have since nhentai does not have a public API so we need to scrape it and make it work out based on what us programmers can manage.
## how to use?
`pip install -r ".\requirements.txt"`
@ -11,6 +22,21 @@ open `nfavorites.py` and it will close and generate set.yaml
open `set.yaml` and enter your cookie and useragent
final open nfavorites.py again and it will generate csv file if everything is ok
## how download ?
After completing the steps above run scraper.py which will read your dumped ids and will download the torrent files.
Proceed to open the torrent files enmasse using your favorite torrent client (i use qbittorent)
run scraper.py until you are sure you did not get rate limited and crashed (u will see a stack trace w/ crash info when u get rate limited)
once completed you should run scraper.py one more time to confirm all the ids were downloaded
DO NOT BECOME A BRAINDEAD UTORRENT USER OR BITTORENT USER THEY ARE SHIT TORRENT CLIENTS
Use [qbittorent](https://www.qbittorrent.org/download), install [qbittorent cli](https://github.com/fedarovich/qbittorrent-cli/) read their setup instructions
run `add_torrents.bat` from this repo to autimatically add all the torrents to your qbittorrent client. this process is 100% automated if you apply common sense and use the free and open source qbittorent client and tools.
I am on Windows, I am unable to upload a shellscript that does the same because i have no idea how qbittorent is setup on Linux or if you have other command line options there and if qbittorrent-cli is even required. It shall be trivial for a Linux user to figure that part out since its just a shell script to add all the dumped torrents to your client.
## how to get my cookie?
open <https://nhentai.net/favorites/>

17
add_torrents.bat Normal file
View File

@ -0,0 +1,17 @@
@echo off
setlocal enabledelayedexpansion
:: Set the directory where your torrent files are located
set "TORRENT_DIR=torrents"
:: Change to the directory containing the torrent files
cd /d "%TORRENT_DIR%"
:: Loop through each .torrent file in the directory
for %%f in (*.torrent) do (
echo Adding %%f
qbt torrent add file "%%f"
)
echo All torrents have been added.
pause

View File

@ -135,19 +135,24 @@ def get_torrents():
print("Please edit set.yaml")
exit()
output_dir = "torrents"
os.makedirs(output_dir, exist_ok=True) # Create the directory if it doesn't exist
for url in url_list:
torrent_url = url
# Call wtfcloudflare to download the torrent file
response = wtfcloudflare_t(torrent_url, useragent=useragent, cookie=cookie)
torrent_id = url.split('/')[4] # The ID is in the 4th segment of the URL
torrent_path = os.path.join(output_dir, f"{torrent_id}.torrent")
# Extract the ID from the URL for naming the file
torrent_id = url.split('/')[4] # The ID is in the 4th segment of the URL
# Define the output directory and file name
output_dir = "torrents"
os.makedirs(output_dir, exist_ok=True) # Create the directory if it doesn't exist
torrent_path = os.path.join(output_dir, f"{torrent_id}.torrent")
# Skip downloading if the torrent file already exists
if os.path.exists(torrent_path):
print(f"Torrent file already exists: {torrent_path}")
continue
response = wtfcloudflare_t(torrent_url, useragent=useragent, cookie=cookie)
# Save the torrent file to disk
with open(torrent_path, 'wb') as torrent_file: