Pirate's Booty Mirroring and Rehosting
Amrayu78
Max Romanchenko No rush on this!
Max Romanchenko
Amrayu78 I'm sorry, I somehow overlooked this task.
I've implemented a mirroring feature on PixelDrain. The target folder is downloaded as an archive from Goofile to our server into a temporary folder, then unpacked and uploaded to the root directory on PixelDrain. As a result, the admin receives a link that can be copied and added to a new PB field. So far, this is only available for episodes, but I will make it available for other resources as well.
Please check if this is what you had in mind.
I uploaded two folders (on the screenshots) -
- "Go_Go_Stupid_Woman__NF_"
- "Izakaya_Shinkansen_2__UNEXT_"
There is a bug with PixelDrain that I discovered - subtitle files open in the browser instead of downloading, because their server does not send attachment headers...
I haven't been able to fix this yet (I think I need some kind of proxy, but I don't want to complicate things too much), so just keep that in mind. Gofile sends all the headers correctly, so there is no such problem there.
Amrayu78
Max Romanchenko Thanks for working on this. Just one question about this part:
The target folder is downloaded as an archive from Goofile to our server into a temporary folder, then unpacked and uploaded to the root directory on PixelDrain.
When you say "our server" do you mean the web server where the site is? Will this be an issue if we transfer many files and batches?
Max Romanchenko
Amrayu78 Yes, exactly.
The folder is downloaded as a single file thanks to the Gofile API (as an archive, meaning that even if there are many files in the folder, everything will be downloaded at once). Then we need to temporarily download this archive before uploading it to PixelDrain on our server, where our website is located. This will not create a noticeable load, and then a folder is created on PixelDrain and the files are extracted into it.
Amrayu78
Max Romanchenko Great, thanks for this! I will test it out and let you know how it goes.
Amrayu78
Max Romanchenko One question though... here's a scenario.
for on-going series, i usually upload a new episode every week, what if I want to rehost this? how would that work?
Would I get a new link every time? or would the script create a folder/collection on pixeldrain and upload it there?
Max Romanchenko
Amrayu78 I understand the workflow, thank you. However, this is more like synchronization... The current implementation will not work for this because it does not know anything about changes on Gofile. I need to think about an effective solution to this problem. I will get back to you soon.
Question - is it okay to save folders from Gofile to the Pixeldrain root? It is impossible to recreate the folder structure as it is on Gofile (this would require a large number of API requests to determine parent folders, etc.).
Amrayu78
Max Romanchenko yup, it’s okay to save folders from gofile to pixeldrain root.
Max Romanchenko
Amrayu78 Thank you! Could you please create a test folder on Gofile and send me a shared link? I will use this folder in my tests.
Amrayu78
Max Romanchenko Here you go:
https://gofile.io/d/81655a8c-967b-42dc-a5ef-99c29a479a54
https://gofile.io/d/3bHFbt
Max Romanchenko
Amrayu78 Thanks, I'm working on it.
Max Romanchenko
This is ready, at least in the beta version.
Available for episodes only (for now).
Of course, this is not true synchronization (Gofile does not yet have event triggers that indicate when a folder has been updated),as we could be banned for too many requests from Gofile and Pixeldrain, so I have created several synchronization options:
- A checkbox to enable/disable folder synchronization for each link.
- How often to check the folder on Gofile (default is 6 hours, minimum is 1 hour for the hottest folders). Please let me know if the options need to be adjusted.
- Then the cron handler will check the synchronized links every 5 minutes and will try to process them to fit within the selected synchronization interval.
Please note that the current implementation synchronizes new files with Gofile, but does not delete a file or an entire folder from PixelDrain if they have been deleted from Goofile. If this happens frequently in practice, I will add it.
Next, I plan to add a table of synchronized links to the admin panel so that it is convenient to see them all together and manage them from there as well.
Please check it out and let me know if I missed anything important.
Max Romanchenko
Amrayu78
Okay, the page in the admin panel is ready. Here is the link:
https://drama-otaku.com/wp-admin/tools.php?page=pixeldrain-mirrors
I've also fixed a bug with downloading text subtitle files from Pixeldrain (the files now download instead of opening).
Amrayu78
Max Romanchenko Thanks for implementing this!
I don't think we need to run the sync that often. A new episode is only uploaded once per week.
Also is it possible to sync the whole folder as before?
There's usually 3 scenarios:
- I upload an entire series (all episodes)
- I upload an episode once a week (until its done)
- I delete and reupload a file that is messed up
Max Romanchenko
Amrayu78 Thanks for the clarification.
Full folder mirroring each time would be inefficient. We'd download all files repeatedly even if only one changed, wasting traffic and time. With multiple folders, this becomes gigabytes of unnecessary downloads.
Partial synchronization makes more sense - it's lighter on resources and won't redownload the entire folder each time. The proposed workflo - first time creates a full mirror, then daily checks for changes, new files added or deleted files removed.
You're correct about frequency. Once daily should be sufficient to check for updates. If nothing changed, the script will exit quickly.
One question - do you ever replace/overwrite the same file on Gofile? Currently the system checks by filename only. If you replace a file with the same name, it won't detect the change. In such cases, it would be better to delete and upload with a new filename.
Is this partial sync approach acceptable, or do you need full resync every time?
Perhaps I misunderstood, sorry. Do you want the changes to be automatic, or do you want to manually search for the link on the website and "mirror" it every time you make changes to the folder on GoFile?
Max Romanchenko
Amrayu78 For your 3 scenarios:
- Entire series: One-time mirror, disable auto-sync.
- Weekly episode - enable auto-sync. System checks daily. Syncs only new files (recommended) to avoid re-downloading everything.
- Replace messed up file - enable auto-sync. Current system won't detect same-name replacements (different hash systems). Simple fix: rename the file (episode-01-v2.mkv). Or I can build hash comparison (more complex).
Does renaming work for you for scenario 3?
Amrayu78
Max Romanchenko Okay, thanks for explaining. I understand what you've built now.
But yes, the above seems okay for our purposes.
I'm only worried about the auto-sync. Is this something I have to turn on? Will it put a strain on the system resources?
Max Romanchenko
Amrayu78 Sorry if I caused any confusion earlier. Let me clarify the current workflow for mirroring links to Pixeldrain:
When a link from Gofile is mirrored, a corresponding folder with the same set of files is created on Pixeldrain. After that, you can choose whether this folder should be synchronized or not.
The idea behind this option is that not all links need to be continuously monitored. For example, movies or completed episodes are unlikely to change, so synchronizing them daily would only generate unnecessary API requests. As the number of mirrored folders grows, constant synchronization could become inefficient.
That's why I added the auto‑sync checkbox and a list of mirrored links in the admin panel, so you can enable or disable synchronization for each folder.
One alternative would be to make auto‑synchronization the default and remove the checkbox. However, this would mean that even folders with old or completed content would still be checked daily, which may waste server resources.
I may be missing something in terms of your expectations, so please let me know if you would prefer a different behavior. My goal is to ensure the workflow meets your requirements.
Max Romanchenko
Amrayu78 I've simplified the logic by removing time interval selection and added a "Delete" button in the admin panel that removes both the database record and the corresponding Pixeldrain folder.
The script now performs two-way synchronization: it deletes files from Pixeldrain that don't exist on GoFile, and uploads new files from GoFile to Pixeldrain.
The cron job is currently set to run every 5 minutes (for testing purposes). Once testing is complete, I'll adjust it to run daily.
Please review the entire process and share your feedback.
Amrayu78
Max Romanchenko Thanks Max! I will test this.
Just for clarity sake, let me explain what I understand from this.
- One time bulk uploads: use the Mirror option
- Weekly uploads (on-going series): first mirror (for whatever's there first) then check the auto-sync. Uncheck the auto-sync when the series is done?
- Pixeldrain mirror dashboard - monitors what auto-syncs we have; where the cron is running.
Did I understand it correctly?
Max Romanchenko
Amrayu78 Yes! You've understood correctly.
Max Romanchenko
Amrayu78 Another completely different method is to use a hybrid admin tool (your own file manager) that will automatically upload content to mirrored folder. This means no auto-synchronization or cron jobs. Everything happens in one place. This means that you will need to perform all actions in it (creating a folder, uploading a file, mirroring a folder on PixelDrain) rather than in the GoFile interface.
This is a beta experiment, I don't know how good the idea is, but I gave it a chance. Please try it out.
The basic options are already working.
The file manager can be found at the url:
https://drama-otaku.com/wp-admin/tools.php?page=file-mirror
Mirrored folders will still be displayed here (and also highlighted with a green icon in the file manager):
https://drama-otaku.com/wp-admin/tools.php?page=pixeldrain-mirrors
Amrayu78
Max Romanchenko Oh I like this different method as well. I haven't had time to properly test everything. My contract job just ended and they kept me really busy up until the last minute.
I have more time now to test everything. I'll let you know how this goes.
Thank you!
Max Romanchenko
Amrayu78 Sure, I somehow felt that you were busy. I hope everything is fine with you. I'll be waiting to hear from you.
Amrayu78
Max Romanchenko I tested the mirroring option and it looks like it's working well. I think I'll have to test this for the next couple months, but we can close out this ticket and if any issues arise, I will just file a ticket for it. Thank you again!
Max Romanchenko
Amrayu78 Thank you! For now, I'll keep both options available (frontend mirroring and file manager). It's possible that this may change in the future, and we might leave only one option to avoid overloading the API.
I've made some improvements to the file manager. It can now synchronize the Pixeldrain folder in real time, eliminating the need for periodic synchronization. This works as long as all file operations are performed within the manager.
Let's see how it goes.
Amrayu78
Max Romanchenko Thank you! I don't intend to mirror too often since gofile has been pretty stable these days. I did mirror something the other day because a user reported an issue.