Fix OverflowError: Python int too large to convert to C long raised during annotation label map creation with custom labels.

[briangow]

The patch makes the label‑store dtype explicit (int64) and replaces row‑wise .loc assignment with a keyed, vectorized update.

[bemoody]

Pandas is Greek to me. What is the issue this is trying to solve? What integer value is too large to convert to long? Annotation *type codes* can't be too large; those must be between 1 and 49.

[briangow]

@bemoody this is to fix the test error currently seen in #544. My understanding is that this is a known issue when Pandas tries to unify data types. Here is the explanation (along with the full test error):

WFDB’s own constraints keep label_store small (1–49), so it isn’t your annotation values that are “too big.” The overflow comes from pandas’ internal attempt to unify dtypes when a row assignment ends up behaving like an append under the hood. On some platforms/pandas versions, that path tries to convert Python ints through a C long and can raise OverflowError even if your data are reasonable.

=================================== FAILURES ===================================
____________________________ TestAnnotation.test_3 _____________________________

self = <tests.test_annotation.TestAnnotation testMethod=test_3>

    def test_3(self):
        """
        Annotation file with custom annotation types
    
        Target file created with:
            rdann -r sample-data/1003 -a atr > ann-3
        """
>       annotation = wfdb.rdann("sample-data/1003", "atr")

tests/test_annotation.py:183: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
wfdb/io/annotation.py:2028: in rdann
    annotation.set_label_elements(return_label_elements)
wfdb/io/annotation.py:1408: in set_label_elements
    self.convert_label_attribute(contained_elements[0], e)
wfdb/io/annotation.py:1471: in convert_label_attribute
    label_map = self.create_label_map(inplace=False)
wfdb/io/annotation.py:911: in create_label_map
    label_map.loc[i] = self.custom_labels.loc[i]
.venv/lib/python3.9/site-packages/pandas/core/indexing.py:912: in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
.venv/lib/python3.9/site-packages/pandas/core/indexing.py:1933: in _setitem_with_indexer
    self._setitem_with_indexer_missing(indexer, value)
.venv/lib/python3.9/site-packages/pandas/core/indexing.py:2329: in _setitem_with_indexer_missing
    self.obj._mgr = self.obj._append(value)._mgr
.venv/lib/python3.9/site-packages/pandas/core/frame.py:10581: in _append
    other = row_df.infer_objects(copy=False).rename_axis(
.venv/lib/python3.9/site-packages/pandas/core/generic.py:6910: in infer_objects
    new_mgr = self._mgr.convert(copy=copy)
.venv/lib/python3.9/site-packages/pandas/core/internals/managers.py:466: in convert
    return self.apply("convert", copy=copy, using_cow=using_copy_on_write())
.venv/lib/python3.9/site-packages/pandas/core/internals/managers.py:363: in apply
    applied = getattr(b, f)(**kwargs)
.venv/lib/python3.9/site-packages/pandas/core/internals/blocks.py:656: in convert
    blocks = self.split_and_operate(
.venv/lib/python3.9/site-packages/pandas/core/internals/blocks.py:473: in split_and_operate
    rbs = func(nb, *args, **kwargs)
.venv/lib/python3.9/site-packages/pandas/core/internals/blocks.py:675: in convert
    res_values = lib.maybe_convert_objects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   OverflowError: Python int too large to convert to C long

pandas/_libs/lib.pyx:2619: OverflowError
=========================== short test summary info ============================
FAILED tests/test_annotation.py::TestAnnotation::test_3 - OverflowError: Python int too large to convert to C long
=================== 1 failed, 69 passed in 281.48s (0:04:41) ===================
Error: Process completed with exit code 1.

[bemoody]

The problem seems to be related to NPY_PROMOTION_STATE=weak_and_warn; perhaps it is the same or similar to pandas-dev/pandas#60023 (which was closed, understandably, because this was never meant to be supported long-term.)

I'd suggest that we've gotten what benefit we can out of NPY_PROMOTION_STATE, and we can probably just remove it from test.yml. Or we could leave it set only for the test-deb11-i386 job.

As for create_label_map, without understanding this code I'd rather not change it. I'd much rather eliminate the use of pandas and rewrite the logic using standard Python data structures and/or numpy.

[briangow]

@bemoody , thanks for looking into this. It does seem like it is probably related to pandas-dev/pandas#60023. I'd be fine with removing NPY_PROMOTION_STATE but if you prefer to keep it only for test-deb11-i386 that is fine also. I can update this PR once you let me know your thoughts.

[briangow]

This was resolved with #552