commit 12bb37ef0a430ab30d0ec3e06d7f58d42b1332b9 Author: Dylan <dylanpdx@gmail.com> Date: Mon Sep 12 17:31:22 2022 +0100 Increase memory to 1GB commit bbdd0f85c500413f765b44cdb8643539a01b3968 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 8 23:19:41 2022 +0100 Reduced memory to 256mb commit 5fa80151a09092a7206ffd294ef464650b776ba8 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 8 23:10:57 2022 +0100 Updated serverless code commit 42f826908ac82a91e67312ae6270ce6ac9bfd216 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 8 19:33:39 2022 +0100 Squashed commit of the following: commit27accd7a5a
Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 23:54:15 2022 +0100 Documented apiMirrors commitbf46166b02
Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 22:58:53 2022 +0100 Fixed incorrect log print 😅 commita05ff596e0
Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 22:52:20 2022 +0100 Fix #14 commit1954a72bd0
Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 22:33:07 2022 +0100 API Mirror commitc58e9b2755
Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 21:38:42 2022 +0100 Updated requirements commita51876a829
Author: Dylan <dylanpdx@gmail.com> Date: Sat Jul 23 16:34:36 2022 +0100 Fix twitter api failing for direct video linking commitd97f70b224
Author: Dylan <dylanpdx@gmail.com> Date: Sat Jul 23 16:29:39 2022 +0100 Added option to disable cache commit2da2cd5200
Author: Dylan <dylanpdx@gmail.com> Date: Fri Jul 22 00:57:07 2022 +0100 Removed image scale-down at the end of processing commitb0547ce810
Author: Dylan <dylanpdx@gmail.com> Date: Sun Jul 17 14:00:19 2022 +0100 Same fix as last commit commit1ec59bce2e
Author: Dylan <dylanpdx@gmail.com> Date: Sun Jul 17 13:54:59 2022 +0100 Fixed direct linking on Discord commit2a77ff96b0
Author: Dylan <dylanpdx@gmail.com> Date: Wed Jul 6 19:43:22 2022 +0100 Separated code into different files commit973c31f705
Merge:e28789a
b27296e
Author: Dylan <dylanpdx@gmail.com> Date: Thu Jun 9 14:35:33 2022 +0100 Merge pull request #7 from flifloo/main Add Docker image and compose with documentation commitb27296e317
Author: flifloo <flifloo@gmail.com> Date: Tue Jun 7 11:46:39 2022 +0200 Add Docker image and compose with documentation commit40f4770e17
Author: flifloo <flifloo@gmail.com> Date: Tue Jun 7 10:44:52 2022 +0200 Fix links.json creation commite28789a09d
Author: Dylan <dylanpdx@gmail.com> Date: Sun Jun 5 13:36:18 2022 +0100 Added og:description on videos for Telegram users commitfaed7482f4
Author: Dylan <dylanpdx@gmail.com> Date: Sun Jun 5 02:10:54 2022 +0100 Update requirements.txt commita5b2eb6cd0
Author: Dylan <dylanpdx@gmail.com> Date: Sat Jun 4 13:53:14 2022 +0100 Restore old behavior of showing error message commit16dfc509d3
Author: Dylan <dylanpdx@gmail.com> Date: Fri Jun 3 23:27:33 2022 +0100 Small log message when an unofficial API call succeeds commit174a20d896
Author: Dylan <dylanpdx@gmail.com> Date: Thu Jun 2 17:07:48 2022 +0100 Use unofficial API when official API fails commite64dee9349
Author: Dylan <dylanpdx@gmail.com> Date: Thu Jun 2 01:09:04 2022 +0100 Temporary redirect for failing links due to API ratelimiting commitf0f709f62e
Author: Dylan <dylanpdx@gmail.com> Date: Wed Jun 1 21:52:47 2022 +0100 use yt-dlp commit0f77b2e0a4
Merge:285c8f7
ffcde09
Author: Dylan <dylanpdx@gmail.com> Date: Wed Jun 1 19:26:22 2022 +0000 Merge branch 'main' of github.com:dylanpdx/BetterTwitFix commit285c8f70f8
Author: Dylan <dylanpdx@gmail.com> Date: Wed Jun 1 19:25:23 2022 +0000 Service auto-restart commitffcde09b97
Author: Dylan <dylanpdx@gmail.com> Date: Mon May 30 01:25:53 2022 +0100 Return HTTP 400 if missing query parameters commit891db049af
Author: Dylan <dylanpdx@gmail.com> Date: Fri May 27 23:18:58 2022 +0100 Fixed incorrect TTL being set commit797b670d0e
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 18:11:30 2022 +0100 Do not combine single image tweets commit728785278b
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 16:48:24 2022 +0100 Fixed direct video linking commitec1f4dc284
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 16:13:31 2022 +0100 Image check in lambda handler commit77b972a1b2
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 16:10:29 2022 +0100 Switched to using ThreadPoolExecutor commit4feb1b65d6
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 15:01:12 2022 +0100 Optimized image downloads commit2819b504a3
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 02:01:43 2022 +0100 Add quality argument commit2399788fdc
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 01:29:47 2022 +0100 Misc code cleanup; Fixed Discord not embedding 301 redirects issue commit7a021916f0
Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 00:58:28 2022 +0100 Use proper mime type commita06a89825c
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 23:56:48 2022 +0100 Fixed formatting commit6717fd464a
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 23:55:41 2022 +0100 Documented combination_method commit022edb5122
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 23:48:55 2022 +0100 Moved combineImg to own module; created AWS dockerfile; added combination_method config commitd7e0cb9089
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 17:28:03 2022 +0100 Changed scaling method commit55ea554357
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 16:30:51 2022 +0100 Re-scale image down commitb457760fae
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 01:20:27 2022 +0100 Use JPEG for final combined img commit39a9ffd512
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 00:49:22 2022 +0100 Method to embed combined tweet images. WIP & may be CPU intensive commita6efd26447
Author: Dylan <dylanpdx@gmail.com> Date: Mon May 23 20:33:11 2022 +0100 Oops- Updated other two error msgs commite81beb975c
Author: Dylan <dylanpdx@gmail.com> Date: Mon May 23 19:02:54 2022 +0100 Improved Fail link message commit55c956ad4e
Author: Dylan <dylanpdx@gmail.com> Date: Sun May 22 15:19:28 2022 +0100 Error checking for loading cache json file commit014f9d0e8d
Author: Dylan <dylanpdx@gmail.com> Date: Sun May 22 15:19:17 2022 +0100 TTL for cache entries commitf640ac69a1
Merge:7eff745
08d686c
Author: Dylan <dylanpdx@gmail.com> Date: Sat May 21 16:13:26 2022 +0100 Merge pull request #2 from cabiste69/patch-1 Minor cleaning commit08d686c652
Author: cabiste <ahmedjj354@gmail.com> Date: Fri May 20 23:14:58 2022 +0100 minor cleaning commit7eff745550
Author: Dylan <dylanpdx@gmail.com> Date: Wed May 18 18:23:11 2022 +0100 Clarifications in readme commit8df211090c
Merge:5a2a4e6
681f191
Author: Dylan <dylanpdx@gmail.com> Date: Tue May 17 17:14:30 2022 +0100 Merge pull request #1 from Fam0r/main Use the best bitrate video for twitter embeds commit5a2a4e6316
Author: Dylan <dylanpdx@gmail.com> Date: Tue May 17 14:55:49 2022 +0100 Display how many total images in the post commit681f191502
Author: Fam0r <fam0r@mailbox.org> Date: Tue May 17 09:40:40 2022 +0300 Use the best bitrate video for twitter embeds commit82fe66af48
Author: Dylan <dylanpdx@gmail.com> Date: Mon May 16 20:18:18 2022 +0100 Work in progress code for combining up to 4 images commitfba1256efd
Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:42:22 2022 +0100 Fixed infinite redirect(?) commit85c682ab11
Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:15:15 2022 +0100 Updated gitignore commit f2eefbe5a542f1a6fa8404228c2a9425054765a8 Author: Dylan <dylanpdx@gmail.com> Date: Wed May 18 18:21:00 2022 +0100 Fixed DynamoDB integration commit 32e5376b987c732147ab3289c2c42a8afde19e2f Author: Dylan <dylanpdx@gmail.com> Date: Mon May 16 19:28:31 2022 +0100 DynamoDB support & Serverless config update commit dee62f812a6c8da443b111c7f3dd24bcb4d76525 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 17:06:42 2022 +0100 Updated serverless.yml to dockerize pip packages commit 43d69a1d872c293a4b70d2a330261245783eb924 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 14:30:18 2022 +0100 serverless wsgi files commit 96246aa9217774ddc88275ffba5bf1790317eda9 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:47:20 2022 +0100 Squashed commit of the following: commitfba1256efd
Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:42:22 2022 +0100 Fixed infinite redirect(?) commit85c682ab11
Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:15:15 2022 +0100 Updated gitignore commit 2c9563fafe97bfe7782386aaa04372dcac5f8136 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:14:41 2022 +0100 Serverless config
vxTwitter
(A fork of TwitFix) Basic flask server that serves fixed twitter video embeds to desktop discord by using either the Twitter API or Youtube-DL to grab tweet video information. This also automatically embeds the first link in the text of non video tweets (API Only)
Differences from fxtwitter
fxtwitter exposed all recently processed tweets publicly via a "latest" and "top" page.
Even though Tweets are public, it was a personal concern for me that a tweet with potentially sensitive information in it could suddenly be shown to however many people were browsing the latest tweets page, and could be used as a tool for harassment. This was removed in The following commit
How to use (discord side)
just put the url to the server, and directly after, the full URL to the tweet you want to embed
I now have a copy of this running on a Linode server, you can use it via the following url
https://vxtwitter.com/[twitter video url] or [last half of twitter url] (everything past twitter.com/)
You can also simply type out 'vx' directly before 'twitter.com' in any valid twitter video url, and that will convert it into a working vxTwitter url, for example:
Note: If you enjoy this service, please considering donating via Ko-Fi to help cover server costs
I do not monitor any tweets processed by this server. Additionally, if you plan on hosting the code yourself and are concerned about this, be sure to check how to disable logging on the web server you are using (i.e Nginx)
How to run (server side)
this script uses the youtube-dl python module, along with flask, twitter and pymongo, so install those with pip (you can use pip install -r requirements.txt
) and start the server with python twitfix.py
I have included some files to give you a head start on setting this server up with uWSGI, though if you decide to use uWSGI I suggest you set up mongoDB link caching
Config
vxTwitter generates a config.json in its root directory the first time you run it, the options are:
API - This will be where you put the credentials for your twitter API if you use this method
database - This is where you put the URL to your mongoDB database if you are using one
link_cache - (Options: db, json)
- db: Caches all links to a mongoDB database. This should be used it you are using uWSGI and are not just running the script on its own as one worker
- json: This saves cached links to a local links.json file
- none: Does not cache requests. Not reccomended as you can easily use up your Twitter API credits with this. Intended for use with another cache system (i.e NGINX uwsgi_cache)
method - ( Options: youtube-dl, api, hybrid )
-
youtube-dl: the original method for grabbing twitter video links, this uses a guest token provided via youtube-dl and should work well for individual instances, but may not scale up to a very large amount of usage
-
api: this directly uses the twitter API to grab tweet info, limited to 900 calls per 15m
-
hybrid: This will start off by using the twitter API to grab tweet info, but if the rate limit is reached or the api fails for any other reason it will switch over to youtube-dl to avoid downtime
color - Accepts a hex formatted color code, can change the embed color
appname - Can change the app name easily wherever it's shown
repo - used to change the repo url that some links redirect to
url - used to tell the user where to look for the oembed endpoint, make sure to set this to your public facing url
combination_method - using c.vxtwitter as the url causes vxTwitter to combine all images in the post into one. This is CPU intensive, so you might not want it running on the same machine that's serving requests. When combination_method
is set to local
, it will use the local machine to combine the images. This requires pillow to be installed. If you want to use another server, replace local
with the URL to the endpoint which combines images. Both methods use the code in the combineImg
module. Inside, there's also a Dockerfile
intended to be deployed as a combination endpoint on an AWS Lambda function.
apiMirrors - During an influx of traffic (i.e when Twitter embeds break!) it's very likely that Twitter will begin to block you for sending too many requests. This is an array of replacement Twitter API URLs which can be called upon when requesting tweet info locally fails. An example of an implementation of an API Mirror is in the twExtract directory, and is intended to be hosted on AWS Lambda. If multiple mirror URLs are specified, one will be selected at random.
This project is licensed under the Do What The Fuck You Want Public License
Other stuff
We check for t.co links in non video tweets, and if one is found, we direct the discord useragent to embed that link directly, this means that twitter links containing youtube / vimeo links will automatically embed those as if you had just directly linked to that content