Files
Dmitry Rogozhkin 00ef8a78e4 Respect each example requirements and use uv (#1330)
* Add requirements.txt to examples which miss them

Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>

* Update numpy requirement for reinforcement_learning to be <2

Current version of the example requires `numpy<2` otherwise the following
error can be seen:
```
AttributeError: module 'numpy' has no attribute 'bool8'. Did you mean: 'bool'?
```

Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>

* Update torch requirement for time and word examples to be <2.6

Current version of examples require `torch<2.6` otherwise the following
error can be seen:
```
  File "/pytorch/examples/time_sequence_prediction/train.py", line 47, in <module>
    data = torch.load('traindata.pt')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/pytorch/examples/time_sequence_prediction/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1524, in load
    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
```

Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>

* Respect each example requirements and use uv

This commit introduces few changes to CI by modifying `run_*_examples.sh`
and respective github workflows:

* Switched to uv
* Added tearup and teardown stages for tests (`start()` and `stop()` methods
  wrapping up test bodies - these are called automatically)
* Tearup (`start()`) installs example dependencies and, optionally (if `VIRTUAL_ENV=.venv`
  is passed), creates uv virtual environment
* Teardown (`stop()`) removes uv virtual environment if it was created (to
  save space)
* If no `VIRTUAL_ENV` set, then scripts expect to be executed in the existing
  virtual environment. These can be `python -m venv`, `uv env` or `conda env`.
  In this case example dependencies will be installed in this environment
  potentially reinstalling existing packages (including `torch`!).
* Dropped automated detection of CUDA platform. Now scripts require `USE_CUDA=True`
  to be passed explicitly
* Added `PIP_INSTALL_ARGS` environment variable to be passed to `uv pip install` calls
  for each example dependencies. This allows to adjust torch indices and other options.

Execute all tests in current virtual environment (might rewrite packages):
```
./run_distributed_examples.sh
```

Execute all tests creating separate environment for each example:
```
VIRTUAL_ENV=.venv ./run_distributed_examples.sh
```

Run with CUDA:
```
USE_CUDA=True ./run_distributed_examples.sh
```

Adjust index:
```
PIP_INSTALL_ARGS="--pre -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html" \
   ./run_distributed_examples.sh
```

Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>

---------

Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
2025-04-25 21:28:00 -07:00

40 lines
1.0 KiB
YAML