commit 12bb37ef0a430ab30d0ec3e06d7f58d42b1332b9 Author: Dylan <dylanpdx@gmail.com> Date: Mon Sep 12 17:31:22 2022 +0100 Increase memory to 1GB commit bbdd0f85c500413f765b44cdb8643539a01b3968 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 8 23:19:41 2022 +0100 Reduced memory to 256mb commit 5fa80151a09092a7206ffd294ef464650b776ba8 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 8 23:10:57 2022 +0100 Updated serverless code commit 42f826908ac82a91e67312ae6270ce6ac9bfd216 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 8 19:33:39 2022 +0100 Squashed commit of the following: commit 27accd7a5a601bd77a3c13832e92ebbd0571b7b5 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 23:54:15 2022 +0100 Documented apiMirrors commit bf46166b027133719ebbd5d1ed46dcae8d242190 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 22:58:53 2022 +0100 Fixed incorrect log print 😅 commit a05ff596e057c774a77fa20381370a1baa60cf20 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 22:52:20 2022 +0100 Fix #14 commit 1954a72bd070d213f539c8a8260b6de9c98a0a59 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 22:33:07 2022 +0100 API Mirror commit c58e9b27554d00c4545a70f62a9b5a7c132563d5 Author: Dylan <dylanpdx@gmail.com> Date: Thu Sep 1 21:38:42 2022 +0100 Updated requirements commit a51876a82943567107ef9c777cfa5ffd9c89f060 Author: Dylan <dylanpdx@gmail.com> Date: Sat Jul 23 16:34:36 2022 +0100 Fix twitter api failing for direct video linking commit d97f70b22494899417cc5fc8744e1d4ea6d22f9e Author: Dylan <dylanpdx@gmail.com> Date: Sat Jul 23 16:29:39 2022 +0100 Added option to disable cache commit 2da2cd52005ac7e7a2827a56cfb894b2294a769c Author: Dylan <dylanpdx@gmail.com> Date: Fri Jul 22 00:57:07 2022 +0100 Removed image scale-down at the end of processing commit b0547ce8107979bac0b32946490f1a4e578cb8fb Author: Dylan <dylanpdx@gmail.com> Date: Sun Jul 17 14:00:19 2022 +0100 Same fix as last commit commit 1ec59bce2ea8a6543854fe24e4eeb738f4499794 Author: Dylan <dylanpdx@gmail.com> Date: Sun Jul 17 13:54:59 2022 +0100 Fixed direct linking on Discord commit 2a77ff96b05dabc916ad4b516897d68477c5d41f Author: Dylan <dylanpdx@gmail.com> Date: Wed Jul 6 19:43:22 2022 +0100 Separated code into different files commit 973c31f705f8e5f86416e3607c8c75fe6c0cc1c1 Merge: e28789a b27296e Author: Dylan <dylanpdx@gmail.com> Date: Thu Jun 9 14:35:33 2022 +0100 Merge pull request #7 from flifloo/main Add Docker image and compose with documentation commit b27296e317e0eec87245fba2a58a38fdcb09e8ec Author: flifloo <flifloo@gmail.com> Date: Tue Jun 7 11:46:39 2022 +0200 Add Docker image and compose with documentation commit 40f4770e177e4003f12758b9f75c3373e56562c2 Author: flifloo <flifloo@gmail.com> Date: Tue Jun 7 10:44:52 2022 +0200 Fix links.json creation commit e28789a09dbb6efd36827a3fd47e0132666a9609 Author: Dylan <dylanpdx@gmail.com> Date: Sun Jun 5 13:36:18 2022 +0100 Added og:description on videos for Telegram users commit faed7482f4b3d09a5656bf5d2a078018bd223e94 Author: Dylan <dylanpdx@gmail.com> Date: Sun Jun 5 02:10:54 2022 +0100 Update requirements.txt commit a5b2eb6cd0472d83a8b0f37ddfa2b358cb5d31f4 Author: Dylan <dylanpdx@gmail.com> Date: Sat Jun 4 13:53:14 2022 +0100 Restore old behavior of showing error message commit 16dfc509d32fe6667a43f9008daf9031bf93a1c4 Author: Dylan <dylanpdx@gmail.com> Date: Fri Jun 3 23:27:33 2022 +0100 Small log message when an unofficial API call succeeds commit 174a20d8961728f45041de00ae1b8e3c30fb2d5d Author: Dylan <dylanpdx@gmail.com> Date: Thu Jun 2 17:07:48 2022 +0100 Use unofficial API when official API fails commit e64dee93498620b0c95b305d72178f23e506dfb6 Author: Dylan <dylanpdx@gmail.com> Date: Thu Jun 2 01:09:04 2022 +0100 Temporary redirect for failing links due to API ratelimiting commit f0f709f62ec40dd3138bde5cff7b027f82d82eb6 Author: Dylan <dylanpdx@gmail.com> Date: Wed Jun 1 21:52:47 2022 +0100 use yt-dlp commit 0f77b2e0a466a09dabbc4eff33b4ce8517fc8148 Merge: 285c8f7 ffcde09 Author: Dylan <dylanpdx@gmail.com> Date: Wed Jun 1 19:26:22 2022 +0000 Merge branch 'main' of github.com:dylanpdx/BetterTwitFix commit 285c8f70f84d5efe96d50a9e239c7e460be5cd38 Author: Dylan <dylanpdx@gmail.com> Date: Wed Jun 1 19:25:23 2022 +0000 Service auto-restart commit ffcde09b976dd85630da73cb4c804b86c63fb760 Author: Dylan <dylanpdx@gmail.com> Date: Mon May 30 01:25:53 2022 +0100 Return HTTP 400 if missing query parameters commit 891db049af0de4d40df3eaabe67d12952a036224 Author: Dylan <dylanpdx@gmail.com> Date: Fri May 27 23:18:58 2022 +0100 Fixed incorrect TTL being set commit 797b670d0e968075cb98de37b1439c59c4470caa Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 18:11:30 2022 +0100 Do not combine single image tweets commit 728785278b549d467e51e6f8fb4492bc5db71cf9 Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 16:48:24 2022 +0100 Fixed direct video linking commit ec1f4dc2845956e4237610e29ebad65b544ea095 Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 16:13:31 2022 +0100 Image check in lambda handler commit 77b972a1b2069e039c5067eb7dd2e95bcb2acea1 Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 16:10:29 2022 +0100 Switched to using ThreadPoolExecutor commit 4feb1b65d6ec0fcff2854b7080f8995e6ae77fe1 Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 15:01:12 2022 +0100 Optimized image downloads commit 2819b504a36f7b39d61640b87cb0a81e804e36a7 Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 02:01:43 2022 +0100 Add quality argument commit 2399788fdc0d8a10063f30c448462e9caa79997f Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 01:29:47 2022 +0100 Misc code cleanup; Fixed Discord not embedding 301 redirects issue commit 7a021916f0c9e8404646104258394e764080bffc Author: Dylan <dylanpdx@gmail.com> Date: Thu May 26 00:58:28 2022 +0100 Use proper mime type commit a06a89825cf29d07787286d4837dfcb7a0151760 Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 23:56:48 2022 +0100 Fixed formatting commit 6717fd464aaa934724ea9c9f6022402013c7238b Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 23:55:41 2022 +0100 Documented combination_method commit 022edb5122ddef8a8b3f84ffe84b1b2e4cf8130f Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 23:48:55 2022 +0100 Moved combineImg to own module; created AWS dockerfile; added combination_method config commit d7e0cb90898ac1aacfbe600f6dd1d8137185791a Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 17:28:03 2022 +0100 Changed scaling method commit 55ea554357aca026926e1d296fb5ac259458c1d3 Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 16:30:51 2022 +0100 Re-scale image down commit b457760fae648c3cad5e383717f4df28c4539303 Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 01:20:27 2022 +0100 Use JPEG for final combined img commit 39a9ffd5120f7d28711e071290ee00d495a477ef Author: Dylan <dylanpdx@gmail.com> Date: Wed May 25 00:49:22 2022 +0100 Method to embed combined tweet images. WIP & may be CPU intensive commit a6efd26447efa9b54e02c48bda9b58b038236ae0 Author: Dylan <dylanpdx@gmail.com> Date: Mon May 23 20:33:11 2022 +0100 Oops- Updated other two error msgs commit e81beb975ce319ccc95fa8db9e3e39927e205929 Author: Dylan <dylanpdx@gmail.com> Date: Mon May 23 19:02:54 2022 +0100 Improved Fail link message commit 55c956ad4ec7fe0ef0cbfb0e740bfcb1783aa88f Author: Dylan <dylanpdx@gmail.com> Date: Sun May 22 15:19:28 2022 +0100 Error checking for loading cache json file commit 014f9d0e8d94da562f834665741adda744c0b42c Author: Dylan <dylanpdx@gmail.com> Date: Sun May 22 15:19:17 2022 +0100 TTL for cache entries commit f640ac69a1ba7f19ad69f6fea2199bec696d5eaf Merge: 7eff745 08d686c Author: Dylan <dylanpdx@gmail.com> Date: Sat May 21 16:13:26 2022 +0100 Merge pull request #2 from cabiste69/patch-1 Minor cleaning commit 08d686c652a5edbf8252771c30a53743b5541f33 Author: cabiste <ahmedjj354@gmail.com> Date: Fri May 20 23:14:58 2022 +0100 minor cleaning commit 7eff745550631e2d0ad014de9e941151bb5a8b61 Author: Dylan <dylanpdx@gmail.com> Date: Wed May 18 18:23:11 2022 +0100 Clarifications in readme commit 8df211090c4d73c812b76dba65c6dc180c848300 Merge: 5a2a4e6 681f191 Author: Dylan <dylanpdx@gmail.com> Date: Tue May 17 17:14:30 2022 +0100 Merge pull request #1 from Fam0r/main Use the best bitrate video for twitter embeds commit 5a2a4e6316dfd1369b7a428ed4c9fcb9fe794de5 Author: Dylan <dylanpdx@gmail.com> Date: Tue May 17 14:55:49 2022 +0100 Display how many total images in the post commit 681f19150219c22a22999ac1e41dc5d393d5b9b5 Author: Fam0r <fam0r@mailbox.org> Date: Tue May 17 09:40:40 2022 +0300 Use the best bitrate video for twitter embeds commit 82fe66af48c5dc4f8d933f2874f8bae815e7dd05 Author: Dylan <dylanpdx@gmail.com> Date: Mon May 16 20:18:18 2022 +0100 Work in progress code for combining up to 4 images commit fba1256efd3ce539f716e2f23b45c122c8422759 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:42:22 2022 +0100 Fixed infinite redirect(?) commit 85c682ab111f575c8c517117f456cf354e9f5bf7 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:15:15 2022 +0100 Updated gitignore commit f2eefbe5a542f1a6fa8404228c2a9425054765a8 Author: Dylan <dylanpdx@gmail.com> Date: Wed May 18 18:21:00 2022 +0100 Fixed DynamoDB integration commit 32e5376b987c732147ab3289c2c42a8afde19e2f Author: Dylan <dylanpdx@gmail.com> Date: Mon May 16 19:28:31 2022 +0100 DynamoDB support & Serverless config update commit dee62f812a6c8da443b111c7f3dd24bcb4d76525 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 17:06:42 2022 +0100 Updated serverless.yml to dockerize pip packages commit 43d69a1d872c293a4b70d2a330261245783eb924 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 14:30:18 2022 +0100 serverless wsgi files commit 96246aa9217774ddc88275ffba5bf1790317eda9 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:47:20 2022 +0100 Squashed commit of the following: commit fba1256efd3ce539f716e2f23b45c122c8422759 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:42:22 2022 +0100 Fixed infinite redirect(?) commit 85c682ab111f575c8c517117f456cf354e9f5bf7 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:15:15 2022 +0100 Updated gitignore commit 2c9563fafe97bfe7782386aaa04372dcac5f8136 Author: Dylan <dylanpdx@gmail.com> Date: Sun May 15 01:14:41 2022 +0100 Serverless config
vxTwitter
(A fork of TwitFix) Basic flask server that serves fixed twitter video embeds to desktop discord by using either the Twitter API or Youtube-DL to grab tweet video information. This also automatically embeds the first link in the text of non video tweets (API Only)
Differences from fxtwitter
fxtwitter exposed all recently processed tweets publicly via a "latest" and "top" page.
Even though Tweets are public, it was a personal concern for me that a tweet with potentially sensitive information in it could suddenly be shown to however many people were browsing the latest tweets page, and could be used as a tool for harassment. This was removed in The following commit
How to use (discord side)
just put the url to the server, and directly after, the full URL to the tweet you want to embed
I now have a copy of this running on a Linode server, you can use it via the following url
https://vxtwitter.com/[twitter video url] or [last half of twitter url] (everything past twitter.com/)
You can also simply type out 'vx' directly before 'twitter.com' in any valid twitter video url, and that will convert it into a working vxTwitter url, for example:
Note: If you enjoy this service, please considering donating via Ko-Fi to help cover server costs
I do not monitor any tweets processed by this server. Additionally, if you plan on hosting the code yourself and are concerned about this, be sure to check how to disable logging on the web server you are using (i.e Nginx)
How to run (server side)
this script uses the youtube-dl python module, along with flask, twitter and pymongo, so install those with pip (you can use pip install -r requirements.txt
) and start the server with python twitfix.py
I have included some files to give you a head start on setting this server up with uWSGI, though if you decide to use uWSGI I suggest you set up mongoDB link caching
Config
vxTwitter generates a config.json in its root directory the first time you run it, the options are:
API - This will be where you put the credentials for your twitter API if you use this method
database - This is where you put the URL to your mongoDB database if you are using one
link_cache - (Options: db, json)
- db: Caches all links to a mongoDB database. This should be used it you are using uWSGI and are not just running the script on its own as one worker
- json: This saves cached links to a local links.json file
- none: Does not cache requests. Not reccomended as you can easily use up your Twitter API credits with this. Intended for use with another cache system (i.e NGINX uwsgi_cache)
method - ( Options: youtube-dl, api, hybrid )
-
youtube-dl: the original method for grabbing twitter video links, this uses a guest token provided via youtube-dl and should work well for individual instances, but may not scale up to a very large amount of usage
-
api: this directly uses the twitter API to grab tweet info, limited to 900 calls per 15m
-
hybrid: This will start off by using the twitter API to grab tweet info, but if the rate limit is reached or the api fails for any other reason it will switch over to youtube-dl to avoid downtime
color - Accepts a hex formatted color code, can change the embed color
appname - Can change the app name easily wherever it's shown
repo - used to change the repo url that some links redirect to
url - used to tell the user where to look for the oembed endpoint, make sure to set this to your public facing url
combination_method - using c.vxtwitter as the url causes vxTwitter to combine all images in the post into one. This is CPU intensive, so you might not want it running on the same machine that's serving requests. When combination_method
is set to local
, it will use the local machine to combine the images. This requires pillow to be installed. If you want to use another server, replace local
with the URL to the endpoint which combines images. Both methods use the code in the combineImg
module. Inside, there's also a Dockerfile
intended to be deployed as a combination endpoint on an AWS Lambda function.
apiMirrors - During an influx of traffic (i.e when Twitter embeds break!) it's very likely that Twitter will begin to block you for sending too many requests. This is an array of replacement Twitter API URLs which can be called upon when requesting tweet info locally fails. An example of an implementation of an API Mirror is in the twExtract directory, and is intended to be hosted on AWS Lambda. If multiple mirror URLs are specified, one will be selected at random.
This project is licensed under the Do What The Fuck You Want Public License
Other stuff
We check for t.co links in non video tweets, and if one is found, we direct the discord useragent to embed that link directly, this means that twitter links containing youtube / vimeo links will automatically embed those as if you had just directly linked to that content