1
0
Fork 0
Commit Graph

75 Commits

Author SHA1 Message Date
Lucidiot a6f7e2a9ce
Fix lxml requirement 2023-01-31 17:11:03 +01:00
Lucidiot 1e73949ec4
Let me have multiple dotenvs 2023-01-31 17:10:43 +01:00
Lucidiot bac5dde8b5
Do not crash on missing token 2022-07-20 10:30:47 +02:00
Lucidiot 39a94276d5
Add auto-retry on 429 when listing emojis for export 2022-07-20 10:18:35 +02:00
Lucidiot 75f3e2220e
Fix API token fetching 2022-07-20 10:18:32 +02:00
Ash Wilson e335cd48bb
Merge pull request #44 from smashwilson/dependabot/pip/lxml-4.6.3
Bump lxml from 4.6.2 to 4.6.3
2021-03-22 13:13:11 -04:00
dependabot[bot] 38e24b34dc
Bump lxml from 4.6.2 to 4.6.3
Bumps [lxml](https://github.com/lxml/lxml) from 4.6.2 to 4.6.3.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.6.2...lxml-4.6.3)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-22 16:58:34 +00:00
Ash Wilson 0f925d85b6
Merge pull request #42 from smashwilson/dependabot/pip/lxml-4.6.2 2021-01-07 17:57:44 -05:00
dependabot[bot] ccb4840cee
Bump lxml from 4.5.0 to 4.6.2
Bumps [lxml](https://github.com/lxml/lxml) from 4.5.0 to 4.6.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.5.0...lxml-4.6.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-01-07 21:56:46 +00:00
Ash Wilson 55bc410833
Merge pull request #40 from aphexcx/aphexcx/files_from_dir_and_api_token_input
Accept dirs as well as single files on command line, and also accept api_token via input if scraping it fails.
2020-08-27 08:20:40 -04:00
afik_cohen 68ecaab488 Accept dirs as well as single files on cmdline, and also accept api_token via input if scraping it fails. 2020-05-15 16:25:06 -07:00
afik_cohen 599e19967d Bump lxml versions to ones that work with python 3.7+. 2020-05-15 16:24:00 -07:00
Ash Wilson ee06c5d9de
Merge pull request #38 from chief-tyrol/tyrol/issues-37
Mark idna-ssl as a requirement (resolves #37)
2019-11-25 10:06:36 -05:00
Tyrol 12a3bd30b3 Mark idna-ssl as a requirement (resolves #37) 2019-11-24 22:17:45 -06:00
Ash Wilson fb8c6c0c65
Merge pull request #36 from chief-tyrol/tyrol/issue-27
Update export.py script to handle new Slack emoji page (resolves #27)
2019-11-18 22:16:53 -05:00
Tyrol ea44a2a453 Update export.py script to handle new Slack emoji page (resolves #27) 2019-11-18 20:44:40 -06:00
Ash Wilson 1c2b5aae3f
Merge pull request #34 from smashwilson/dependabot/pip/requests-2.20.0
Bump requests from 2.18.4 to 2.20.0
2019-10-18 11:20:16 -04:00
dependabot[bot] 9c4cc1db88
Bump requests from 2.18.4 to 2.20.0
Bumps [requests](https://github.com/requests/requests) from 2.18.4 to 2.20.0.
- [Release notes](https://github.com/requests/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/master/HISTORY.md)
- [Commits](https://github.com/requests/requests/compare/v2.18.4...v2.20.0)

Signed-off-by: dependabot[bot] <support@github.com>
2019-10-18 14:34:05 +00:00
Ash Wilson cbc44fa0b8
Merge pull request #33 from manos/retry-after
honor HTTP retry-after header for better success
2019-09-04 14:08:24 -04:00
Charlie Schluting edf0dd3a9c honor HTTP retry-after header for better success 2019-09-04 12:54:14 -05:00
Ash Wilson 4cb36752ee
Merge pull request #32 from denbeigh2000/master
fixed scraping for slack ui changes, improve failure messaging, add backoff
2019-09-02 20:08:14 -04:00
Denbeigh Stevens fdd77eb853 added exponential backoff for rate limiting 2019-09-02 11:40:30 -07:00
Denbeigh Stevens 0d773f2d68 fixed scraping for slack ui changes, improve failure messaging 2019-09-02 11:04:49 -07:00
Ash Wilson fbcf759ebb
Merge pull request #29 from matsu-chara/apply-slack-new-ui
update upload.py for slack new ui
2018-08-28 08:52:02 -04:00
matsu-chara 48cc567b80 remove redundunt logs 2018-08-28 14:33:14 +09:00
matsu-chara 415ab46398 update upload.py for slack new ui 2018-08-28 14:31:27 +09:00
David Lord 4f98dc188e
Merge pull request #23 from smashwilson/pagination-fix
Follow pagination for >500 emoji
2018-06-21 13:43:21 +10:00
David Lord af453e6f39 Coerce `--concurrent-requests`/`-r` to type int
It comes through from the environment variable as a string. Fixes this
unhelpful error message:

`TypeError: '<' not supported between instances of 'str' and 'int'`
2018-06-12 19:50:36 +10:00
David Lord 25a3405192 Follow pagination for >500 emoji
At some point, Slack started paginating on the export page for teams with huge emoji collections.
This means the export process would only detect the 500 emoji on that first page (ordered alphabetically).
It now looks to other pages and exports those as well, restoring full functionality.

Tested on a team with ~1300 emoji (3 pages).
2018-06-12 19:17:59 +10:00
Ash Wilson f4dfff6fd9
Merge pull request #21 from smashwilson/aiohttp
aiohttp export
2018-01-24 10:50:42 -05:00
David Lord 8bfcb401a3 Add aiohttp to Pipfile 2018-01-24 16:54:03 +11:00
David Lord 2d4e355849 Initial implementation of aiohttp export
Makes the export script require Python 3.6.
(async/await were added in 3.5, so maybe there?)

I'll need to tidy it up a bit, but this is v1 of functionality.

Adds -r/--concurrent-requests for throttling. Defaults to 200.
(configured in .env as CONCURRENT_REQUESTS)

Caveats:
- Adds an aiohttp dependency.
- Stops using upload._session, effectively duplicating the
functionality to get access to aiohttp.ClientSession.
- Adds logging to record the files downloaded. Previously silent.

I've also noticed a bug in filename parsing where a bunch of files all
named `apple.png` are created. This script parses the URL to retrieve
the filename, which exposes this duplication. My version does more
parsing in the HTML to detect the :emoji_name: as used by Slack clients.
Currently I'm not addressing this.
2017-10-25 14:20:28 +10:00
Ash Wilson a9f3ea6059 Fix pipenv run commands 2017-10-20 21:11:03 -04:00
Ash Wilson ec9bf59b47 Bring Pipefile and Pipfile.lock up to date 2017-10-20 21:06:24 -04:00
Ash Wilson e0ef751878 Merge branch 'pipenv' 2017-10-20 21:04:55 -04:00
Ash Wilson 2e2588175e Merge branch 'pr-14' 2017-10-20 21:03:34 -04:00
Ash Wilson 7b0cedb0f1 Typo fix 2017-10-20 21:02:02 -04:00
Ash Wilson 62335e34d7 Missing letter 2017-10-20 20:59:32 -04:00
Ash Wilson 2060f4fc87 Change the README to use pipenv 2017-10-20 20:53:50 -04:00
Ash Wilson c1d5701d0e Use a * dependency on slacker like pipenv warned me 2017-10-20 20:53:27 -04:00
Ash Wilson d21516bb31 Pipfile 2017-10-20 20:47:47 -04:00
Ash Wilson ebb422f5a1 Tiny typo fix 2017-10-20 20:47:33 -04:00
Ash Wilson cf42b84133 Merge pull request #17 from abn/emoji-name
Support emoji name prefix/suffix
2017-07-24 11:12:31 -04:00
Arun Babu Neelicattu 60aa4c8653 Support emoji name prefix/suffix
- add support for prefixes via --prefix/$EMOJI_NAME_PREFIX
- add support for suffixes via --suffix/$EMOJI_NAME_SUFFIX
2017-07-25 01:34:51 +12:00
Ash Wilson 20b1737876 Merge pull request #16 from ZhilinS/patch-1
A little fix to upload.py
2017-05-18 10:06:27 -04:00
Sergey Zhilin fffc1ac8b0 A little fix to upload.py
get_current_emoji_list function had a greedy quantifier in regexp: "data-emoji-name=\"(.*)\""
I.e. for string like
`data-emoji-name="parrot" data-action="emoji.remove"`
it returns following result:
`parrot" data-action="emoji.remove`

In case of lazy quantifier ("data-emoji-name=\"(.*?)\"") the result will be correct:`parrot`
2017-05-18 16:07:16 +03:00
Stephen Brennan 5666828a1f Note that libxml is required for bulk export. 2017-04-26 10:33:03 -04:00
Stephen Brennan 6f901efc64 Remove obsolete environment variables. 2017-03-21 23:54:04 -04:00
Stephen Brennan 73d1f64f04 Update README for exporting. 2017-03-21 23:50:53 -04:00
Stephen Brennan 0ab95cdfe1 Require lxml, no more slacker. 2017-03-21 23:48:56 -04:00