Dylan 10ec46a0cf Squashed commit of the following:
commit 12bb37ef0a430ab30d0ec3e06d7f58d42b1332b9
Author: Dylan <dylanpdx@gmail.com>
Date:   Mon Sep 12 17:31:22 2022 +0100

    Increase memory to 1GB

commit bbdd0f85c500413f765b44cdb8643539a01b3968
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu Sep 8 23:19:41 2022 +0100

    Reduced memory to 256mb

commit 5fa80151a09092a7206ffd294ef464650b776ba8
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu Sep 8 23:10:57 2022 +0100

    Updated serverless code

commit 42f826908ac82a91e67312ae6270ce6ac9bfd216
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu Sep 8 19:33:39 2022 +0100

    Squashed commit of the following:

    commit 27accd7a5a
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Sep 1 23:54:15 2022 +0100

        Documented apiMirrors

    commit bf46166b02
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Sep 1 22:58:53 2022 +0100

        Fixed incorrect log print 😅

    commit a05ff596e0
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Sep 1 22:52:20 2022 +0100

        Fix #14

    commit 1954a72bd0
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Sep 1 22:33:07 2022 +0100

        API Mirror

    commit c58e9b2755
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Sep 1 21:38:42 2022 +0100

        Updated requirements

    commit a51876a829
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sat Jul 23 16:34:36 2022 +0100

        Fix twitter api failing for direct video linking

    commit d97f70b224
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sat Jul 23 16:29:39 2022 +0100

        Added option to disable cache

    commit 2da2cd5200
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Fri Jul 22 00:57:07 2022 +0100

        Removed image scale-down at the end of processing

    commit b0547ce810
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun Jul 17 14:00:19 2022 +0100

        Same fix as last commit

    commit 1ec59bce2e
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun Jul 17 13:54:59 2022 +0100

        Fixed direct linking on Discord

    commit 2a77ff96b0
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed Jul 6 19:43:22 2022 +0100

        Separated code into different files

    commit 973c31f705
    Merge: e28789a b27296e
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Jun 9 14:35:33 2022 +0100

        Merge pull request #7 from flifloo/main

        Add Docker image and compose with documentation

    commit b27296e317
    Author: flifloo <flifloo@gmail.com>
    Date:   Tue Jun 7 11:46:39 2022 +0200

        Add Docker image and compose with documentation

    commit 40f4770e17
    Author: flifloo <flifloo@gmail.com>
    Date:   Tue Jun 7 10:44:52 2022 +0200

        Fix links.json creation

    commit e28789a09d
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun Jun 5 13:36:18 2022 +0100

        Added og:description on videos for Telegram users

    commit faed7482f4
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun Jun 5 02:10:54 2022 +0100

        Update requirements.txt

    commit a5b2eb6cd0
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sat Jun 4 13:53:14 2022 +0100

        Restore old behavior of showing error message

    commit 16dfc509d3
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Fri Jun 3 23:27:33 2022 +0100

        Small log message when an unofficial API call succeeds

    commit 174a20d896
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Jun 2 17:07:48 2022 +0100

        Use unofficial API when official API fails

    commit e64dee9349
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu Jun 2 01:09:04 2022 +0100

        Temporary redirect for failing links due to API ratelimiting

    commit f0f709f62e
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed Jun 1 21:52:47 2022 +0100

        use yt-dlp

    commit 0f77b2e0a4
    Merge: 285c8f7 ffcde09
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed Jun 1 19:26:22 2022 +0000

        Merge branch 'main' of github.com:dylanpdx/BetterTwitFix

    commit 285c8f70f8
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed Jun 1 19:25:23 2022 +0000

        Service auto-restart

    commit ffcde09b97
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Mon May 30 01:25:53 2022 +0100

        Return HTTP 400 if missing query parameters

    commit 891db049af
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Fri May 27 23:18:58 2022 +0100

        Fixed incorrect TTL being set

    commit 797b670d0e
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 18:11:30 2022 +0100

        Do not combine single image tweets

    commit 728785278b
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 16:48:24 2022 +0100

        Fixed direct video linking

    commit ec1f4dc284
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 16:13:31 2022 +0100

        Image check in lambda handler

    commit 77b972a1b2
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 16:10:29 2022 +0100

        Switched to using ThreadPoolExecutor

    commit 4feb1b65d6
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 15:01:12 2022 +0100

        Optimized image downloads

    commit 2819b504a3
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 02:01:43 2022 +0100

        Add quality argument

    commit 2399788fdc
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 01:29:47 2022 +0100

        Misc code cleanup; Fixed Discord not embedding 301 redirects issue

    commit 7a021916f0
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Thu May 26 00:58:28 2022 +0100

        Use proper mime type

    commit a06a89825c
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 23:56:48 2022 +0100

        Fixed formatting

    commit 6717fd464a
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 23:55:41 2022 +0100

        Documented combination_method

    commit 022edb5122
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 23:48:55 2022 +0100

        Moved combineImg to own module; created AWS dockerfile; added combination_method config

    commit d7e0cb9089
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 17:28:03 2022 +0100

        Changed scaling method

    commit 55ea554357
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 16:30:51 2022 +0100

        Re-scale image down

    commit b457760fae
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 01:20:27 2022 +0100

        Use JPEG for final combined img

    commit 39a9ffd512
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 25 00:49:22 2022 +0100

        Method to embed combined tweet images. WIP & may be CPU intensive

    commit a6efd26447
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Mon May 23 20:33:11 2022 +0100

        Oops- Updated other two error msgs

    commit e81beb975c
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Mon May 23 19:02:54 2022 +0100

        Improved Fail link message

    commit 55c956ad4e
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun May 22 15:19:28 2022 +0100

        Error checking for loading cache json file

    commit 014f9d0e8d
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun May 22 15:19:17 2022 +0100

        TTL for cache entries

    commit f640ac69a1
    Merge: 7eff745 08d686c
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sat May 21 16:13:26 2022 +0100

        Merge pull request #2 from cabiste69/patch-1

        Minor cleaning

    commit 08d686c652
    Author: cabiste <ahmedjj354@gmail.com>
    Date:   Fri May 20 23:14:58 2022 +0100

        minor cleaning

    commit 7eff745550
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Wed May 18 18:23:11 2022 +0100

        Clarifications in readme

    commit 8df211090c
    Merge: 5a2a4e6 681f191
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Tue May 17 17:14:30 2022 +0100

        Merge pull request #1 from Fam0r/main

        Use the best bitrate video for twitter embeds

    commit 5a2a4e6316
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Tue May 17 14:55:49 2022 +0100

        Display how many total images in the post

    commit 681f191502
    Author: Fam0r <fam0r@mailbox.org>
    Date:   Tue May 17 09:40:40 2022 +0300

        Use the best bitrate video for twitter embeds

    commit 82fe66af48
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Mon May 16 20:18:18 2022 +0100

        Work in progress code for combining up to 4 images

    commit fba1256efd
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun May 15 01:42:22 2022 +0100

        Fixed infinite redirect(?)

    commit 85c682ab11
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun May 15 01:15:15 2022 +0100

        Updated gitignore

commit f2eefbe5a542f1a6fa8404228c2a9425054765a8
Author: Dylan <dylanpdx@gmail.com>
Date:   Wed May 18 18:21:00 2022 +0100

    Fixed DynamoDB integration

commit 32e5376b987c732147ab3289c2c42a8afde19e2f
Author: Dylan <dylanpdx@gmail.com>
Date:   Mon May 16 19:28:31 2022 +0100

    DynamoDB support & Serverless config update

commit dee62f812a6c8da443b111c7f3dd24bcb4d76525
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun May 15 17:06:42 2022 +0100

    Updated serverless.yml to dockerize pip packages

commit 43d69a1d872c293a4b70d2a330261245783eb924
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun May 15 14:30:18 2022 +0100

    serverless wsgi files

commit 96246aa9217774ddc88275ffba5bf1790317eda9
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun May 15 01:47:20 2022 +0100

    Squashed commit of the following:

    commit fba1256efd
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun May 15 01:42:22 2022 +0100

        Fixed infinite redirect(?)

    commit 85c682ab11
    Author: Dylan <dylanpdx@gmail.com>
    Date:   Sun May 15 01:15:15 2022 +0100

        Updated gitignore

commit 2c9563fafe97bfe7782386aaa04372dcac5f8136
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun May 15 01:14:41 2022 +0100

    Serverless config
2022-09-12 17:36:11 +01:00
2022-09-01 22:33:07 +01:00
2022-09-12 17:36:11 +01:00
2021-07-03 18:52:30 -05:00
2022-09-12 17:36:11 +01:00
2022-09-01 23:54:15 +01:00
2022-09-01 21:38:42 +01:00
2022-05-13 17:00:16 +00:00
2022-09-12 17:36:11 +01:00
2022-06-01 19:25:23 +00:00

vxTwitter

(A fork of TwitFix) Basic flask server that serves fixed twitter video embeds to desktop discord by using either the Twitter API or Youtube-DL to grab tweet video information. This also automatically embeds the first link in the text of non video tweets (API Only)

Differences from fxtwitter

fxtwitter exposed all recently processed tweets publicly via a "latest" and "top" page.

Even though Tweets are public, it was a personal concern for me that a tweet with potentially sensitive information in it could suddenly be shown to however many people were browsing the latest tweets page, and could be used as a tool for harassment. This was removed in The following commit

How to use (discord side)

just put the url to the server, and directly after, the full URL to the tweet you want to embed

I now have a copy of this running on a Linode server, you can use it via the following url

https://vxtwitter.com/[twitter video url] or [last half of twitter url] (everything past twitter.com/)

You can also simply type out 'vx' directly before 'twitter.com' in any valid twitter video url, and that will convert it into a working vxTwitter url, for example:

Note: If you enjoy this service, please considering donating via Ko-Fi to help cover server costs

I do not monitor any tweets processed by this server. Additionally, if you plan on hosting the code yourself and are concerned about this, be sure to check how to disable logging on the web server you are using (i.e Nginx)

How to run (server side)

this script uses the youtube-dl python module, along with flask, twitter and pymongo, so install those with pip (you can use pip install -r requirements.txt) and start the server with python twitfix.py

I have included some files to give you a head start on setting this server up with uWSGI, though if you decide to use uWSGI I suggest you set up mongoDB link caching

Config

vxTwitter generates a config.json in its root directory the first time you run it, the options are:

API - This will be where you put the credentials for your twitter API if you use this method

database - This is where you put the URL to your mongoDB database if you are using one

link_cache - (Options: db, json)

  • db: Caches all links to a mongoDB database. This should be used it you are using uWSGI and are not just running the script on its own as one worker
  • json: This saves cached links to a local links.json file
  • none: Does not cache requests. Not reccomended as you can easily use up your Twitter API credits with this. Intended for use with another cache system (i.e NGINX uwsgi_cache)

method - ( Options: youtube-dl, api, hybrid )

  • youtube-dl: the original method for grabbing twitter video links, this uses a guest token provided via youtube-dl and should work well for individual instances, but may not scale up to a very large amount of usage

  • api: this directly uses the twitter API to grab tweet info, limited to 900 calls per 15m

  • hybrid: This will start off by using the twitter API to grab tweet info, but if the rate limit is reached or the api fails for any other reason it will switch over to youtube-dl to avoid downtime

color - Accepts a hex formatted color code, can change the embed color

appname - Can change the app name easily wherever it's shown

repo - used to change the repo url that some links redirect to

url - used to tell the user where to look for the oembed endpoint, make sure to set this to your public facing url

combination_method - using c.vxtwitter as the url causes vxTwitter to combine all images in the post into one. This is CPU intensive, so you might not want it running on the same machine that's serving requests. When combination_method is set to local, it will use the local machine to combine the images. This requires pillow to be installed. If you want to use another server, replace local with the URL to the endpoint which combines images. Both methods use the code in the combineImg module. Inside, there's also a Dockerfile intended to be deployed as a combination endpoint on an AWS Lambda function.

apiMirrors - During an influx of traffic (i.e when Twitter embeds break!) it's very likely that Twitter will begin to block you for sending too many requests. This is an array of replacement Twitter API URLs which can be called upon when requesting tweet info locally fails. An example of an implementation of an API Mirror is in the twExtract directory, and is intended to be hosted on AWS Lambda. If multiple mirror URLs are specified, one will be selected at random.

This project is licensed under the Do What The Fuck You Want Public License

Other stuff

We check for t.co links in non video tweets, and if one is found, we direct the discord useragent to embed that link directly, this means that twitter links containing youtube / vimeo links will automatically embed those as if you had just directly linked to that content

Description
Our fork of BetterTwitFix
Readme 15 MiB
Languages
Python 84.7%
HTML 9.1%
CSS 2.6%
Dockerfile 1.9%
Shell 1%
Other 0.7%