|
|
9cb12880cc
|
Updated to 72 hour retention
|
2024-02-17 12:59:49 +00:00 |
|
|
|
f0a70605da
|
Update telegram_automated_cdx_processor.py
|
2024-02-05 08:44:50 +00:00 |
|
|
|
a82c328401
|
Add telegram_automated_cdx_processor.py
|
2024-02-05 08:40:29 +00:00 |
|
|
|
badcd7d432
|
Add blogger_automated_cdx_processor.py
|
2024-01-18 23:39:28 +00:00 |
|
|
|
10b7b6f65a
|
Add youtube_automated_cdx_processor.py
|
2024-01-13 01:24:29 +00:00 |
|
|
|
d6d8721c1b
|
Update imgur_automated_cdx_processor.py
|
2024-01-10 00:44:41 +00:00 |
|
|
|
05fd72705e
|
Update pastebin_automated_cdx_processor.py
|
2024-01-10 00:44:01 +00:00 |
|
|
|
261fcfafda
|
Update imgur_automated_cdx_processor.py
|
2024-01-10 00:43:16 +00:00 |
|
|
|
81bd092284
|
Update pastebin_automated_cdx_processor.py
|
2024-01-10 00:42:57 +00:00 |
|
|
|
0c062e7326
|
Update mediafire_automated_cdx_processor.py
|
2024-01-10 00:31:54 +00:00 |
|
|
|
d3d110789a
|
Updated bug in code for previous change
|
2024-01-09 22:48:26 +00:00 |
|
|
|
43d55a27fd
|
Set ia path variable for cron execution
|
2024-01-09 21:35:25 +00:00 |
|
|
|
6c16ee45bd
|
Updated cdx file retention to 2 days
|
2024-01-09 21:15:18 +00:00 |
|
|
|
1e44917569
|
Updated cdx file retention to 2 days
|
2024-01-09 21:14:23 +00:00 |
|
|
|
8e1912cb9b
|
Updated cdx file retention to 2 days
|
2024-01-09 21:13:48 +00:00 |
|
|
|
4d4dc9c81c
|
Add imgur_automated_cdx_processor.py
|
2024-01-09 05:51:41 +00:00 |
|
|
|
08dc0c0bf6
|
Removing the tar process and now deleting cdx.json files older than 24 hours
|
2024-01-09 05:49:56 +00:00 |
|
|
|
51c127601a
|
Removing the tar process and now deleting cdx.json files older than 24 hours
|
2024-01-09 05:45:25 +00:00 |
|
|
|
49a9bc4710
|
Add pastebin_automated_cdx_processor.py
|
2024-01-09 05:15:31 +00:00 |
|
|
|
457f32ed61
|
Added tar support for files older than 24 hours to keep cdx file count low
|
2024-01-09 00:19:22 +00:00 |
|
|
|
f5928594a5
|
Changed time delta to 1 day to minimize cdx file count
|
2024-01-08 23:50:17 +00:00 |
|
|
|
85319ba390
|
Updated code to move files to /opt/cdxfiles/mediafire
|
2024-01-08 11:09:35 +00:00 |
|
|
|
42be737ee6
|
Code fix
|
2024-01-08 11:05:07 +00:00 |
|
|
|
2bf2e02275
|
Update mediafire_automated_cdx_processor.py
|
2024-01-08 10:44:33 +00:00 |
|
|
|
d56d7bb81f
|
Added in support to compare already existing files to skip them being reprocessed
|
2024-01-08 10:37:24 +00:00 |
|
|
|
61f5a4ddbf
|
Update concurrency of downloads
|
2024-01-08 08:27:15 +00:00 |
|
|
|
2e938624f9
|
Add mediafire_automated_cdx_processor.py
|
2024-01-08 08:23:28 +00:00 |
|
|
|
b2a9c44c9e
|
Added in a cdx filesize count statistic for the Grafana Dashboard to report on sizing
|
2023-12-27 08:58:44 +00:00 |
|
|
|
2dddda28bf
|
Add into a "cdxcount": 1, count for metric reporting of processed CDX files
|
2023-12-27 08:52:23 +00:00 |
|
|
|
182c58f1ce
|
Implemented subprocess for running multiple json extractions at once
|
2023-12-27 08:46:36 +00:00 |
|
|
|
8496391064
|
Update to remove tophosts output
|
2023-12-26 11:59:44 +00:00 |
|
|
|
ae515fb425
|
Upload files to "/"
|
2023-12-23 04:39:00 +00:00 |
|
