Troubleshooting
Troubleshooting guide¶
SpectralBridge is designed to be restart-safe, so the first recovery step is usually to rerun the same command and let validated stages skip instead of starting over manually.
Preserve artifacts
Keep QA outputs, correction JSON, and the exact command so failures stay diagnosable.
Reduce pressure
Lower worker counts and chunk sizes before assuming the workflow logic is wrong.
Prefer reruns
Completed outputs are validated and reused when they are still good.
General failures
When the pipeline stops mid-run¶
Common causes include out-of-memory conditions, full temporary directories, unexpected NEON file structure, or interrupted network access during download.
Recommended responses:
- reduce
--max-workers - lower
--parquet-chunk-size - set
CSCAL_TMPDIRto a larger scratch disk - rerun the same
spectralbridge-pipelinecommand so completed stages can be reused
Input issues
Download and HDF5 access problems¶
Missing or corrupted HDF5 files
Verify the site, year-month, product code, and flight line identifier. If a prior download was interrupted, rerun the pipeline so the download stage can validate and rebuild the file if needed.
Repeated validation failure
If the same partial file keeps failing, move it aside or start from a fresh working directory rather than editing the file in place.
Export stage
HDF5 to ENVI export issues¶
ENVI header recognition problems or missing wavelengths usually point to malformed metadata, older NEON metadata layout differences, or interrupted writes.
Recommended responses:
- confirm the HDF5 product is
DP1.30006.001 - confirm the filename includes
directional_reflectance - remove or archive the invalid output pair, then rerun the same pipeline command
For single-flightline debugging, keep the execution path simple:
spectralbridge-pipeline ... --engine thread --max-workers 1
Correction stage
Topographic and BRDF correction problems¶
Dark or clipped areas after correction
Check for deep shadows, DEM mismatch, unstable slope or aspect values, or unusual solar and view geometry. Compare the raw and corrected products in the QA outputs before changing logic.
NaNs or extreme BRDF values
Inspect correction JSON and QA JSON diagnostics. Low-SNR bands or unstable geometry can make coefficient fitting noisy. Preserve the failing artifacts for review rather than silently changing coefficients.
Harmonization stage
Sensor translation issues¶
Landsat-style reflectance looks wrong
Check wavelength metadata, FWHM values, and QA diagnostics before assuming the sensor response definitions are wrong. Problems often propagate from earlier correction stages.
Brightness coefficients are unusually large
This can indicate poor alignment between corrected spectra and the target sensor response frame. Review reflectance scaling, BRDF stability, and the brightness plots in QA.
Parquet and QA
Output and reporting problems¶
Memory errors during extraction or merge
Reduce --max-workers, lower --parquet-chunk-size, tune --merge-memory-limit, and point --merge-temp-directory at a large local scratch disk.
Corrupted parquet sidecars
Rerun the same pipeline command. Export stages validate parquet outputs and rebuild invalid sidecars from ENVI sources when needed.
Missing QA PNG, PDF, or JSON
The run may have stopped before QA rendering or before the merged parquet was available. Rerun the pipeline for the same flight line first.
QA-only rerender
If the upstream products are already present, rerender QA directly:
spectralbridge-qa --base-folder output_demo
Escalation
What to preserve when asking for help¶
Persistent issues are much easier to diagnose when you keep:
- the exact command and environment details
- the flight line identifier
- the QA PNG, PDF, and JSON artifacts
- the correction JSON
- a small metadata snippet from the relevant ENVI header
Where to go next