Skip to content

fix: prevent data loss during currency parsing - #538

Open
alibro005 wants to merge 1 commit into
FreshCode-Org:mainfrom
alibro005:fix/currency-parsing-data-loss
Open

alibro005 wants to merge 1 commit into
FreshCode-Org:mainfrom
alibro005:fix/currency-parsing-data-loss

Conversation

@alibro005

Copy link
Copy Markdown
Contributor

What changed?

  • Added checks to validate values before converting a column to currency values.
  • Prevented identifiers and other non-currency strings from being incorrectly converted.
  • Skipped conversion when a column contains invalid values.
  • Added a regression test to make sure identifiers are preserved when mixed with currency values.

Why?

Previously, the currency parser removed non-numeric characters before checking whether a value was actually a currency. This could cause values like REF-9921 to be converted into -9921.

This change makes sure that columns containing non-currency values are left unchanged instead of being incorrectly converted.

Fixes #533

How was it tested?

  • Added a regression test in tests/test_stages.py.
  • Ran the full test suite using pytest.
  • Ran the fast CI test lane using pytest -m "not online and not large".
  • Ran Ruff and mypy checks.

Any performance impact?

  • None expected. The validation adds only a small amount of processing during currency parsing.

Any compatibility concerns?

  • No public API changes. Existing valid currency columns continue to be parsed as before.

Documentation updated?

  • No documentation changes were needed for this internal bug fix.

@coderabbitai

coderabbitai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 982e6704-4384-4573-8d28-41caf7a9e0e5
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(stages): Prevent silent data loss during currency parsing in stage_dtypes

1 participant