ArchiveTeam/Migrated_ArchiveOrg_CDX_Stats_Processor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Added in a cdx filesize count statistic for the Grafana Dashboard to report on sizing

Browse Source
This commit is contained in:
datechnoman 2023-12-27 08:58:44 +00:00
parent 2dddda28bf
commit b2a9c44c9e
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
mediafire_cdxprocessor.py
View File

@ -22,6 +22,10 @@ def run_cdxsummary(file_path, json_filepath):
# Add "cdxcount" entry with value 1 # Add "cdxcount" entry with value 1
json_output["cdxcount"] = 1 json_output["cdxcount"] = 1
# Add "cdxsize" entry with the size of the cdx.gz file in bytes
cdx_size_bytes = os.path.getsize(file_path)
json_output["cdxsize"] = cdx_size_bytes
# Remove "pathquery" and "samples" keys # Remove "pathquery" and "samples" keys
if "pathquery" in json_output: if "pathquery" in json_output:
del json_output["pathquery"] del json_output["pathquery"]