Commit Graph

Select branches

Hide Pull Requests

main

#1

Mono Color

• 319d65735c Update README.md main datechnoman 2024-03-31 11:55:02 +0000
• 7c157c5a48 Update warc_wat_url_processor.py datechnoman 2024-03-31 11:45:33 +0000
• f95513b9fd Update README.md datechnoman 2024-03-31 11:44:01 +0000
• 511629b012 Update README.md datechnoman 2024-03-31 11:40:22 +0000
• c2d5ad43b8 Delete urlextractor_archiveteam.sh datechnoman 2024-03-31 11:39:22 +0000
• 8b7607c7e6 Update prerequisites.sh datechnoman 2024-03-31 11:38:19 +0000
• a882482f52 Delete rsyncd.conf datechnoman 2024-03-31 11:38:06 +0000
• 1849c03cdf Delete rsyncd.passwd datechnoman 2024-03-31 11:38:01 +0000
• d90c139807 revert b33c8148a4 datechnoman 2024-03-31 11:35:57 +0000
• b33c8148a4 Upload files to "test" datechnoman 2024-03-31 11:34:52 +0000
• 8ecbfc8696 Add archive_org_url_processor.py datechnoman 2024-02-15 11:29:58 +0000
• 722838b24a Updated to include zstd package datechnoman 2024-02-13 22:35:10 +0000
• 3151fce353 Update warc_wat_url_processor.py datechnoman 2024-02-08 00:39:09 +0000
• 486a68a796 Removed multithread compression and added force overwrite for compression files datechnoman 2024-01-28 22:50:09 +0000
• ebc07a6974 Update zstd to overwrite conflicts datechnoman 2024-01-28 11:39:00 +0000
• 29d24e9826 Reverting datechnoman 2024-01-28 11:36:17 +0000
• 6d591ef0d0 Added in error logging datechnoman 2024-01-28 11:29:53 +0000
• 54747b64f6 Update warc_wat_url_processor.py datechnoman 2024-01-28 11:23:27 +0000
• b6a9c68140 Update warc_wat_url_processor.py datechnoman 2024-01-28 09:14:12 +0000
• d0fa7c84f4 Update warc_wat_url_processor.py datechnoman 2024-01-28 09:10:17 +0000
• 875082e8d9 Update prerequisites.sh datechnoman 2024-01-27 06:31:36 +0000
• ea8aa8f755 Update commoncrawl_local_to_share_move.ps1 datechnoman 2024-01-27 06:28:44 +0000
• 54d85523b4 Update commoncrawl_transfer.ps1 datechnoman 2024-01-27 06:28:16 +0000
• 0014256679 Update commoncrawl_transfer.ps1 datechnoman 2024-01-27 06:03:49 +0000
• 4f46c841b3 Updated to include disk space checking datechnoman 2024-01-27 03:02:54 +0000
• 488571b4c0 Added in Lock file support datechnoman 2024-01-27 02:42:11 +0000
• 674a7ae450 Update to zst extension datechnoman 2024-01-27 02:26:27 +0000
• 7eafadc2fe Update to zst file extension datechnoman 2024-01-27 02:24:36 +0000
• 8244b9241a Update concurrency datechnoman 2024-01-26 08:01:39 +0000
• f76dbb13fd Update README.md datechnoman 2024-01-26 07:17:23 +0000
• 5661ce44b2 Updated loop issue datechnoman 2024-01-26 07:14:25 +0000
• 2bee6db85a Updated error processing for zstd compression errors datechnoman 2024-01-25 01:33:11 +0000
• 097ec759ab Update prerequisites.sh datechnoman 2024-01-24 06:00:42 +0000
• 295c3daba4 Update warc_wat_url_processor.py datechnoman 2024-01-23 10:45:32 +0000
• 6edffba451 Update warc_wat_url_processor.py datechnoman 2024-01-23 04:43:15 +0000
• 513b32e80a Update warc_wat_url_processor.py datechnoman 2024-01-20 12:33:28 +0000
• 06e3399861 Update warc_wat_url_processor.py datechnoman 2024-01-20 12:28:23 +0000
• 3ce09f46d7 Update prerequisites.sh datechnoman 2024-01-20 11:30:43 +0000
• e98f80aec4 Updated to use command line zstd datechnoman 2024-01-20 11:29:57 +0000
• 78f6b69cdf Updated with higher compression datechnoman 2024-01-20 11:25:59 +0000
• d1cfd0178f Update prerequisites.sh datechnoman 2024-01-20 11:14:42 +0000
• 17b4ce6077 Update warc_wat_url_processor.py datechnoman 2024-01-20 03:26:03 +0000
• 99c2f07498 Add warc_wat_url_processor.py datechnoman 2024-01-20 03:25:50 +0000
• b881641c69 Commented out URL Extractions (to be done post downloading of files) datechnoman 2024-01-12 04:02:06 +0000
• cba96e96e7 Rollback of change datechnoman 2023-12-22 01:06:30 +0000
• bfc13cb6ef Updated script to keep regenerating a list of files to download datechnoman 2023-12-21 09:35:39 +0000
• 50e89b9de2 Add in checking for new files once list is depleted datechnoman 2023-12-21 01:58:55 +0000
• 0aad853966 Update README.md datechnoman 2023-12-20 04:09:47 +0000
• 5f152307f2 Add commoncrawl_local_to_share_move.ps1 datechnoman 2023-12-20 04:09:13 +0000
• fd9376cbe0 Updated to extract Pastebin URL's datechnoman 2023-12-19 00:23:55 +0000
• 1036de64a7 Documentation Update datechnoman 2023-12-18 04:34:20 +0000
• 727d2c3187 Update commoncrawl_transfer.ps1 datechnoman 2023-12-18 04:27:30 +0000
• 171d3e2d2d Upload files to "/" datechnoman 2023-12-18 04:27:03 +0000
• 65757f8cc4 Update README.md datechnoman 2023-12-12 11:04:52 +0000
• 1e25ef86fe Upload files to "/" datechnoman 2023-12-12 10:23:58 +0000
• b24287ef6f Upload files to "/" datechnoman 2023-12-12 10:23:37 +0000
• b7ce7aa4b0 Upload files to "/" datechnoman 2023-12-12 10:22:31 +0000
• 7f9480bc40 Update urlextractor_archiveteam.sh datechnoman 2023-12-12 10:22:22 +0000
• ac0f299269 Update commoncrawl_url_processor.py datechnoman 2023-12-12 10:05:46 +0000
• ba11c6af9f Update commoncrawl_url_processor.py datechnoman 2023-12-12 10:05:23 +0000
• d97491b4f0 Update urlextractor_archiveteam.sh datechnoman 2023-12-10 05:10:10 +0000
• 6a90fc7f5e Update multithread_script_5.0.py datechnoman 2023-12-06 10:59:41 +0000
• adc15ec579 Update urlextractor_archiveteam.sh datechnoman 2023-12-06 10:35:31 +0000
• 0a3cfd1008 Update urlextractor_archiveteam.sh datechnoman 2023-12-06 10:19:02 +0000
• 807f0136ad Update urlextractor_archiveteam.sh datechnoman 2023-12-05 11:34:29 +0000
• 0d5a54df43 Update urlextractor_archiveteam.sh datechnoman 2023-12-05 10:20:47 +0000
• 805b133dba Update multithread_script_5.0.py datechnoman 2023-12-05 10:20:23 +0000
• bd939a62de Update urlextractor_archiveteam.sh datechnoman 2023-12-05 02:41:43 +0000
• d95927c230 Update multithread_script_5.0.py datechnoman 2023-12-05 02:41:08 +0000
• 00c0678135 Update multithread_script_5.0.py datechnoman 2023-12-04 12:00:27 +0000
• d326a35340 Delete urls_to_download.txt datechnoman 2023-12-04 10:59:19 +0000
• 85529493e3 Delete urls_to_download.txt datechnoman 2023-12-04 10:58:26 +0000
• 6500764baa Update multithread_script_5.0.py datechnoman 2023-12-04 10:53:05 +0000
• dcf47e4085 Upload files to "/" datechnoman 2023-12-04 10:51:34 +0000
• c804a1e1ec Upload files to "/" datechnoman 2023-12-04 10:49:28 +0000
• 8ce7ed8f1a Upload files to "/" datechnoman 2023-12-04 10:48:04 +0000
• 5dd8e4f9ce Upload files to "/" datechnoman 2023-12-04 10:39:04 +0000