|
|
49a9bc4710
|
Add pastebin_automated_cdx_processor.py
|
2024-01-09 05:15:31 +00:00 |
|
|
|
457f32ed61
|
Added tar support for files older than 24 hours to keep cdx file count low
|
2024-01-09 00:19:22 +00:00 |
|
|
|
f5928594a5
|
Changed time delta to 1 day to minimize cdx file count
|
2024-01-08 23:50:17 +00:00 |
|
|
|
85319ba390
|
Updated code to move files to /opt/cdxfiles/mediafire
|
2024-01-08 11:09:35 +00:00 |
|
|
|
42be737ee6
|
Code fix
|
2024-01-08 11:05:07 +00:00 |
|
|
|
2bf2e02275
|
Update mediafire_automated_cdx_processor.py
|
2024-01-08 10:44:33 +00:00 |
|
|
|
d56d7bb81f
|
Added in support to compare already existing files to skip them being reprocessed
|
2024-01-08 10:37:24 +00:00 |
|
|
|
61f5a4ddbf
|
Update concurrency of downloads
|
2024-01-08 08:27:15 +00:00 |
|
|
|
2e938624f9
|
Add mediafire_automated_cdx_processor.py
|
2024-01-08 08:23:28 +00:00 |
|
|
|
b2a9c44c9e
|
Added in a cdx filesize count statistic for the Grafana Dashboard to report on sizing
|
2023-12-27 08:58:44 +00:00 |
|
|
|
2dddda28bf
|
Add into a "cdxcount": 1, count for metric reporting of processed CDX files
|
2023-12-27 08:52:23 +00:00 |
|
|
|
182c58f1ce
|
Implemented subprocess for running multiple json extractions at once
|
2023-12-27 08:46:36 +00:00 |
|
|
|
8496391064
|
Update to remove tophosts output
|
2023-12-26 11:59:44 +00:00 |
|
|
|
ae515fb425
|
Upload files to "/"
|
2023-12-23 04:39:00 +00:00 |
|
