Skip to content

Repair logging statement in find_issue_specific_csv_files #621

@krivard

Description

@krivard

This afternoon's batch issue upload generated the following error:

Traceback (most recent call last):
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/automation/driver/delphi/epidata/acquisition/covidcast/csv_to_database.py", line 222, in <module>
    main(get_argument_parser().parse_args())
  File "/home/automation/driver/delphi/epidata/acquisition/covidcast/csv_to_database.py", line 193, in main
    path_details = collect_files_impl(args.data_dir, args.specific_issue_date)
  File "/home/automation/driver/delphi/epidata/acquisition/covidcast/csv_to_database.py", line 58, in collect_files
    results = list(csv_importer_impl.find_issue_specific_csv_files(data_dir))
  File "/home/automation/driver/delphi/epidata/acquisition/covidcast/csv_importer.py", line 109, in find_issue_specific_csv_files
    logger.info('processing csv files from issue date: "' + str(issue_date) + '", directory', path)
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/site-packages/structlog/stdlib.py", line 145, in info
    return self._proxy_to_logger("info", event, *args, **kw)
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/site-packages/structlog/stdlib.py", line 210, in _proxy_to_logger
    return super()._proxy_to_logger(method_name, event=event, **event_kw)
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/site-packages/structlog/_base.py", line 198, in _proxy_to_logger
    args, kw = self._process_event(method_name, event, event_kw)
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/site-packages/structlog/_base.py", line 155, in _process_event
    event_dict = proc(self._logger, method_name, event_dict)
  File "/home/automation/.pyenv/versions/3.8.2/lib/python3.8/site-packages/structlog/stdlib.py", line 601, in __call__
    event_dict["event"] = event_dict["event"] % args
TypeError: not all arguments converted during string formatting

This is probably a missed keyword from #611, but it's the sort of thing for which we'd rather not rely on my ability to catch runtime errors by eye. We need to:

  • Figure out why unit and integration tests didn't catch it
  • Add tests that fail appropriately
  • Fix the broken code so that the test passes

Batch issue uploads aren't mission-critical, so it's okay if this takes a day or two to repair. If it takes longer than that, let me know so I can adjust the automation schedule.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions