ruk·si

🤖 Machine Learning
Testing

Updated at 2023-12-23 16:50

Unit tests can save weeks of time in machine learning projects. It can take days before you notice that something is not converging properly. And without tests, it can be hard concluding when and where was the issue introduced.

Most frameworks don't validate your neural network architecture. This stems from two the reason; 1) the frameworks are also used for research where new approaches are being explored and 2) even many of them are stable, they are still fairly new, rapidly developed tools.

The most frequent source of bugs is copy-pasting code without understanding. Especially in the non-data science code. Most data scientists are not software engineers, so they frequently utilize online solutions without fully understanding what is happening. You can mitigate this by researching what the different functions actually do, at least on the high level.

Possibly applicable approaches to testing:

  • Check that the network parameters change at least a bit after a mock image training.
  • If you have some network parameters that shouldn't change, check that they don't.
  • Check that loss is never 0%.
  • Check that accuracy is never 100%.
  • Seed your random generator so the tests are deterministic.

Sources