alterware/BetterTwitFix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Squashed commit of the following:

Browse Source 
commit aae42b88c9e193c861648f9a68f871163aa9562d
Merge: f2cec85 ab59881
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu Jun 15 09:16:46 2023 +0100

    Merge branch 'main' of github.com:dylanpdx/BetterTwitFix

commit f2cec8521fef6571596e6f8c3ad32b4409e28b71
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu Jun 15 09:16:41 2023 +0100

    Very very quick fix for API changes

commit ab5988148cd7ffd473d35c686fe4853f0fe01d3a
Merge: 16f133a 1511551
Author: Dylan <dylanpdx@gmail.com>
Date:   Sat Jun 3 15:04:23 2023 +0100

    Merge pull request #98 from dylanpdx/dependabot/pip/boto3-1.26.146

    Bump boto3 from 1.26.133 to 1.26.146

commit 1511551719770cd0a5e34dbc51b94628abc56800
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Sat Jun 3 13:34:51 2023 +0000

    Bump boto3 from 1.26.133 to 1.26.146

    Bumps [boto3](https://github.com/boto/boto3) from 1.26.133 to 1.26.146.
    - [Release notes](https://github.com/boto/boto3/releases)
    - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
    - [Commits](https://github.com/boto/boto3/compare/1.26.133...1.26.146)

    ---
    updated-dependencies:
    - dependency-name: boto3
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 16f133a11bc0b6b4dbdf2ea158900b75954396df
Merge: 9f8f3ea 6b884d4
Author: Dylan <dylanpdx@gmail.com>
Date:   Sat Jun 3 14:34:10 2023 +0100

    Merge pull request #96 from dylanpdx/dependabot/pip/requests-2.31.0

    Bump requests from 2.30.0 to 2.31.0

commit 6b884d42b0b77d10ce3ccb7772f1268bb7581d0f
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Mon May 29 02:00:29 2023 +0000

    Bump requests from 2.30.0 to 2.31.0

    Bumps [requests](https://github.com/psf/requests) from 2.30.0 to 2.31.0.
    - [Release notes](https://github.com/psf/requests/releases)
    - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
    - [Commits](https://github.com/psf/requests/compare/v2.30.0...v2.31.0)

    ---
    updated-dependencies:
    - dependency-name: requests
      dependency-type: direct:production
      update-type: version-update:semver-minor
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 9f8f3ea41e7d1a5ae31232a7968a1c5ef9203bd5
Merge: cda291f a710867
Author: Dylan <dylanpdx@gmail.com>
Date:   Tue May 16 15:44:07 2023 +0100

    Merge branch 'prod' into main

commit cda291fe9eb424ef1e56a450a66ec4766ec554ca
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Tue May 16 15:42:18 2023 +0100

    Bump boto3 from 1.26.129 to 1.26.133 (#92)

    Bumps [boto3](https://github.com/boto/boto3) from 1.26.129 to 1.26.133.
    - [Release notes](https://github.com/boto/boto3/releases)
    - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
    - [Commits](https://github.com/boto/boto3/compare/1.26.129...1.26.133)

    ---
    updated-dependencies:
    - dependency-name: boto3
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

commit 5cb8d479f513a07f0ccb8c41be06095abc37aae9
Author: Dylan <dylanpdx@gmail.com>
Date:   Tue May 16 15:40:34 2023 +0100

    Fixed QRTs sometimes failing to embed

commit b7f285c2cd20e298e4adde743d55a9e45b547c9b
Merge: 66c0044 49ec005
Author: Dylan <dylanpdx@gmail.com>
Date:   Tue May 16 15:32:28 2023 +0100

    Merge branch 'main' of github.com:dylanpdx/BetterTwitFix

commit 66c004453b7134710fac0c692642e489c7c3dbd5
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun May 14 22:04:35 2023 +0100

    Increased timeout to 15

commit 49ec0059cdc7e3812be46d063c260f0247641711
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Tue May 9 16:53:02 2023 +0100

    Bump boto3 from 1.26.127 to 1.26.129 (#90)

    Bumps [boto3](https://github.com/boto/boto3) from 1.26.127 to 1.26.129.
    - [Release notes](https://github.com/boto/boto3/releases)
    - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
    - [Commits](https://github.com/boto/boto3/compare/1.26.127...1.26.129)

    ---
    updated-dependencies:
    - dependency-name: boto3
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

commit c395c00a11ab273607714923f6e8914c3f9e488c
Author: Dylan <dylanpdx@gmail.com>
Date:   Fri May 5 14:08:23 2023 +0100

    Fix text trimming with images and text posts

commit c6840ad093e0d6fe9bcc14f0090ff57de142f568
Author: Dylan <dylanpdx@gmail.com>
Date:   Fri May 5 13:56:45 2023 +0100

    Revert "Fix text trimming w/ old limits"

    This reverts commit 9d9c32b79728aece09fd1fa44625f23ee2b8cb92.

commit 9d9c32b79728aece09fd1fa44625f23ee2b8cb92
Author: Dylan <dylanpdx@gmail.com>
Date:   Fri May 5 13:46:42 2023 +0100

    Fix text trimming w/ old limits

commit dda1187f4a3bf9c4d34fb8cc5ab83ece935ac4ee
Merge: 14c9426 f0065ce
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:48:42 2023 +0100

    Merge branch 'prod' into main

commit 14c942645c1acd2128c1d55b75c16f0159dd351a
Merge: d621bfc e895e4c
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:45:19 2023 +0100

    Merge pull request #84 from dylanpdx/dependabot/pip/requests-2.30.0

    Bump requests from 2.28.2 to 2.30.0

commit e895e4c8762b61ed4e75612a9a55a7768af918de
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu May 4 22:44:08 2023 +0000

    Bump requests from 2.28.2 to 2.30.0

    Bumps [requests](https://github.com/psf/requests) from 2.28.2 to 2.30.0.
    - [Release notes](https://github.com/psf/requests/releases)
    - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
    - [Commits](https://github.com/psf/requests/compare/v2.28.2...v2.30.0)

    ---
    updated-dependencies:
    - dependency-name: requests
      dependency-type: direct:production
      update-type: version-update:semver-minor
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit d621bfc8a8415edda684a51f9245c24ca25fce14
Merge: 727d144 98e006b
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:43:28 2023 +0100

    Merge pull request #86 from dylanpdx/dependabot/pip/boto3-1.26.127

    Bump boto3 from 1.26.104 to 1.26.127

commit 727d144ece9aa9083418fb6e0d9e00db6592df38
Merge: 1a698c1 618a644
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:42:04 2023 +0100

    Merge pull request #87 from dylanpdx/dependabot/npm_and_yarn/simple-git-3.18.0

    Bump simple-git from 3.7.1 to 3.18.0

commit 618a644070094c5b4bfcd37cc468b46eee62ff24
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu May 4 22:40:58 2023 +0000

    Bump simple-git from 3.7.1 to 3.18.0

    Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.7.1 to 3.18.0.
    - [Release notes](https://github.com/steveukx/git-js/releases)
    - [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md)
    - [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.18.0/simple-git)

    ---
    updated-dependencies:
    - dependency-name: simple-git
      dependency-type: indirect
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 98e006b31c4ec11e93bd6cd300d2de1069c8c679
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu May 4 22:39:39 2023 +0000

    Bump boto3 from 1.26.104 to 1.26.127

    Bumps [boto3](https://github.com/boto/boto3) from 1.26.104 to 1.26.127.
    - [Release notes](https://github.com/boto/boto3/releases)
    - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
    - [Commits](https://github.com/boto/boto3/compare/1.26.104...1.26.127)

    ---
    updated-dependencies:
    - dependency-name: boto3
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 1a698c1d6fa591fe1665c28119dad84eb7921697
Merge: a6eb39e 8205ef2
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:38:58 2023 +0100

    Merge pull request #71 from dylanpdx/dependabot/pip/pillow-9.5.0

    Bump pillow from 9.4.0 to 9.5.0

commit a6eb39e5451870f9bf0592190058238c563131a0
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:35:52 2023 +0100

    Fix tweet not found test

commit d84938ee081c4d774abaebc44864a77ea9c5a36b
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:29:12 2023 +0100

    Fixed tweet not found error

commit 69b99dcc9bf8ffee587800d6ccb5cab419f4e115
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:08:11 2023 +0100

    Revert "Add info message for #74"

    This reverts commit 39490a49b98ba6e56a109958dab4cb953ede4fe8.

commit 8674c8d82826106bf7f348c6a574a8f4cb92925f
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 23:07:15 2023 +0100

    Fix/workarounds for failing tests

commit 3127fff1aec40b11598bff898717405949f3c364
Author: Dylan <dylanpdx@gmail.com>
Date:   Thu May 4 22:45:55 2023 +0100

    Update gitignore

commit 6244c875c47061b49bf1516bdfcede1720f13776
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun Apr 9 23:11:31 2023 +0100

    Fix small embed sizes

commit edfeefbc00b90950e76d2726c2a48591f183033f
Author: Dylan <dylanpdx@gmail.com>
Date:   Sun Apr 9 22:26:38 2023 +0100

    Fix NSFW video embeds

commit dd961df082d9333e31ab37e4891f99744e85e2a6
Author: Dylan <dylanpdx@gmail.com>
Date:   Sat Apr 8 18:11:14 2023 +0100

    Update deploy workflow

commit 8205ef2400cafdf6273b1f59d45ef1da081d48aa
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Sat Apr 8 16:59:27 2023 +0000

    Bump pillow from 9.4.0 to 9.5.0

    Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.4.0 to 9.5.0.
    - [Release notes](https://github.com/python-pillow/Pillow/releases)
    - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
    - [Commits](https://github.com/python-pillow/Pillow/compare/9.4.0...9.5.0)

    ---
    updated-dependencies:
    - dependency-name: pillow
      dependency-type: direct:production
      update-type: version-update:semver-minor
    ...

    Signed-off-by: dependabot[bot] <support@github.com>
This commit is contained in:
Dylan 2023-06-15 09:17:39 +01:00
parent 1986b52ea6
commit 6b13c1cd0f
5 changed files with 27 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.github/workflows/deploy.yml vendored
View File

@ -68,4 +68,5 @@ jobs:
VXTWITTER_WORKAROUND_CONSUMER_KEY: ${{ secrets.VXTWITTER_WORKAROUND_CONSUMER_KEY }}
VXTWITTER_WORKAROUND_CONSUMER_SECRET: ${{ secrets.VXTWITTER_WORKAROUND_CONSUMER_SECRET }}
VXTWITTER_WORKAROUND_TOKEN: ${{ secrets.VXTWITTER_WORKAROUND_TOKEN }}
VXTWITTER_WORKAROUND_TOKEN_SECRET: ${{ secrets.VXTWITTER_WORKAROUND_TOKEN_SECRET }}
VXTWITTER_WORKAROUND_TOKEN_SECRET: ${{ secrets.VXTWITTER_WORKAROUND_TOKEN_SECRET }}
VXTWITTER_WORKAROUND_TOKENS: ${{ secrets.VXTWITTER_WORKAROUND_TOKENS }}

1
configHandler.py
View File

@ -15,6 +15,7 @@ elif ('RUNNING_SERVERLESS' in os.environ and os.environ['RUNNING_SERVERLESS'] ==
"url": os.environ["VXTWITTER_URL"],
"combination_method": os.environ["VXTWITTER_COMBINATION_METHOD"], # can either be 'local' or a URL to a server handling requests in the same format
"gifConvertAPI":os.environ["VXTWITTER_GIF_CONVERT_API"],
"workaroundTokens":os.environ["VXTWITTER_WORKAROUND_TOKENS"],
"workaroundKeys":{
"consumerKey":os.environ["VXTWITTER_WORKAROUND_CONSUMER_KEY"],
"consumerSecret":os.environ["VXTWITTER_WORKAROUND_CONSUMER_SECRET"],

4
requirements.txt
View File

@ -1,6 +1,6 @@
pymongo==4.3.3
boto3==1.26.133
requests==2.30.0
boto3==1.26.146
requests==2.31.0
Pillow==9.5.0
Flask==2.2.3
Flask-Cors==3.0.10

1
serverless.yml
View File

@ -31,6 +31,7 @@ provider:
VXTWITTER_WORKAROUND_CONSUMER_SECRET: ${env:VXTWITTER_WORKAROUND_CONSUMER_SECRET, ''}
VXTWITTER_WORKAROUND_TOKEN: ${env:VXTWITTER_WORKAROUND_TOKEN, ''}
VXTWITTER_WORKAROUND_TOKEN_SECRET: ${env:VXTWITTER_WORKAROUND_TOKEN_SECRET, ''}
VXTWITTER_WORKAROUND_TOKENS: ${env:VXTWITTER_WORKAROUND_TOKENS, ''}
package:
patterns:

43
twExtract/__init__.py
View File

@ -1,26 +1,17 @@
import yt_dlp
from yt_dlp.extractor import twitter
import uuid
import json
import requests
import re
import random
from . import twExtractError
import twitter
from configHandler import config
bearer="Bearer AAAAAAAAAAAAAAAAAAAAAPYXBAAAAAAACLXUNDekMxqa8h%2F40K4moUkGsoc%3DTYfbDKbT3jJPCEVnMYqilB28NHfOPqkca3qaAxGfsyKCs0wRbw"
guestToken=None
pathregex = r"\w{1,15}\/(status|statuses)\/(\d{2,20})"
userregex = r"^https?:\/\/(?:www\.)?twitter\.com\/(?:#!\/)?@?([^/?#]*)(?:[?#/].*)?$"
userIDregex = r"\/i\/user\/(\d+)"
try:
auth = twitter.oauth.OAuth(
config['config']['workaroundKeys']["accessToken"],
config['config']['workaroundKeys']["accessTokenSecret"],
config['config']['workaroundKeys']["consumerKey"],
config['config']['workaroundKeys']["consumerSecret"]
)
api = twitter.Twitter(auth=auth)
except Exception as e:
api = None
def getGuestToken():
global guestToken
@ -30,17 +21,25 @@ def getGuestToken():
return guestToken
def extractStatus_fallback(url):
if api is None:
raise twExtractError.TwExtractError(500, "Could not extract tweet.")
print(" ➤ [ I ] Using fallback method to extract tweet")
# get tweet ID
m = re.search(pathregex, url)
if m is None:
raise twExtractError.TwExtractError(400, "Invalid URL")
twid = m.group(2)
# get tweet
tweet = api.statuses.show(_id=twid, tweet_mode="extended")
return tweet
try:
# get tweet ID
m = re.search(pathregex, url)
if m is None:
raise twExtractError.TwExtractError(400, "Extract error")
twid = m.group(2)
# get tweet
authToken=random.choice(config["config"]["workaroundTokens"].split(","))
csrfToken=str(uuid.uuid4()).replace('-', '')
tweet = requests.get("https://api.twitter.com/1.1/statuses/show/" + twid + ".json?tweet_mode=extended&cards_platform=Web-12&include_cards=1&include_reply_count=1&include_user_entities=0", headers={"Authorization":bearer,"Cookie":f"auth_token={authToken}; ct0={csrfToken}; ","x-twitter-active-user":"yes","x-twitter-auth-type":"OAuth2Session","x-twitter-client-language":"en","x-csrf-token":csrfToken,"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/116.0"})
output = tweet.json()
if "errors" in output:
# pick the first error and create a twExtractError
error = output["errors"][0]
raise twExtractError.TwExtractError(error["code"], error["message"])
return output
except Exception as e:
raise twExtractError.TwExtractError(400, "Extract error")
def extractStatus(url):