Dependency_Inversion_Principle_in_Python.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMl/PMeP/5Q2+nm8M1BQdmx",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Tanu-N-Prabhu/Python/blob/master/Dependency_Inversion_Principle_in_Python.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Build Reliable Machine Learning Pipelines with the Dependency Inversion Principle in Python\n",
        "\n",
        "## Decouple your ML components for maximum testability, flexibility, and scalability.\n",
        "\n",
        "\n",
        "| ![space-1.jpg](https://github.com/Tanu-N-Prabhu/Python/blob/master/Img/christina-wocintechchat-com-SqmaKDvcIso-unsplash.jpg?raw=true) |\n",
        "|:--:|\n",
        "|Photo by <a href=\"https://unsplash.com/@wocintechchat?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash\">Christina @ wocintechchat.com</a> on <a href=\"https://unsplash.com/photos/shallow-focus-photo-of-python-book-SqmaKDvcIso?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash\">Unsplash</a>|\n",
        "\n",
        "### Introduction\n",
        "Machine Learning systems aren’t just models, they’re complex software systems with data pipelines, model orchestration, and deployment layers. Yet many ML engineers overlook design principles that make code production-ready. Today, let’s look at the Dependency Inversion Principle (DIP) and how it can transform the way you structure ML systems.\n",
        "\n",
        "---\n",
        "\n",
        "### Problem\n",
        "In typical ML scripts, low-level modules (like Scikit-learn or Pandas) are tightly coupled with high-level business logic. This creates brittle systems; changing one thing breaks everything else. It also makes testing and scaling nearly impossible.\n",
        "\n",
        "---\n",
        "\n",
        "### Design Principle\n",
        "#### Dependency Inversion Principle (DIP)\n",
        "\n",
        "From SOLID principles, DIP states:\n",
        "\n",
        "> *High-level modules should not depend on low-level modules. Both should depend on abstractions.*\n",
        "\n",
        "In ML terms: your training code shouldn’t care whether you use Scikit-learn, XGBoost, or PyTorch — it should depend on abstract interfaces.\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "### Code Implementation (Clean ML Training with DIP)\n",
        "\n"
      ],
      "metadata": {
        "id": "2BO4OAwwDEEk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# interfaces.py\n",
        "from abc import ABC, abstractmethod\n",
        "\n",
        "class IDataLoader(ABC):\n",
        "    @abstractmethod\n",
        "    def load_data(self):\n",
        "        pass\n",
        "\n",
        "class IModel(ABC):\n",
        "    @abstractmethod\n",
        "    def train(self, X, y):\n",
        "        pass\n",
        "\n",
        "    @abstractmethod\n",
        "    def evaluate(self, X, y):\n",
        "        pass"
      ],
      "metadata": {
        "id": "VqsLiwakEIHA"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install interfaces"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2uq244YjEOqS",
        "outputId": "ff78178f-f92b-4482-bcd4-c0d5db270d07"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting interfaces\n",
            "  Downloading interfaces-0.0.4.tar.gz (2.8 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Building wheels for collected packages: interfaces\n",
            "  Building wheel for interfaces (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for interfaces: filename=interfaces-0.0.4-py3-none-any.whl size=3251 sha256=d3d13f45df4d9ebeca2d4fe4ef37ce85746847da0b6fe245a0d87e26e08dda07\n",
            "  Stored in directory: /root/.cache/pip/wheels/9d/f5/73/07bdb84637b7fbdef79552b29797f459815be4e2d37aa2ca61\n",
            "Successfully built interfaces\n",
            "Installing collected packages: interfaces\n",
            "Successfully installed interfaces-0.0.4\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# sklearn_implementations.py\n",
        "from sklearn.datasets import load_iris\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "from interfaces import IDataLoader, IModel\n",
        "\n",
        "class SklearnDataLoader(IDataLoader):\n",
        "    def load_data(self):\n",
        "        data = load_iris()\n",
        "        return data.data, data.target\n",
        "\n",
        "class SklearnRFModel(IModel):\n",
        "    def __init__(self):\n",
        "        self.model = RandomForestClassifier()\n",
        "\n",
        "    def train(self, X, y):\n",
        "        self.model.fit(X, y)\n",
        "\n",
        "    def evaluate(self, X, y):\n",
        "        return accuracy_score(y, self.model.predict(X))"
      ],
      "metadata": {
        "id": "Ocv7pxUQEKt_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# main.py\n",
        "from sklearn_implementations import SklearnDataLoader, SklearnRFModel\n",
        "\n",
        "def run_pipeline(data_loader, model):\n",
        "    X, y = data_loader.load_data()\n",
        "    model.train(X, y)\n",
        "    accuracy = model.evaluate(X, y)\n",
        "    print(\"Model Accuracy:\", accuracy)\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    run_pipeline(SklearnDataLoader(), SklearnRFModel())"
      ],
      "metadata": {
        "id": "cybMUUsjEUw9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Output\n",
        "\n",
        "Model Accuracy: 1.0\n",
        "\n",
        "---\n",
        "\n",
        "### Code Explanation\n",
        "\n",
        "* `interfaces.py`: Defines abstractions for loading data and training models.\n",
        "\n",
        "* `sklearn_implementations.py`: Implements these interfaces using Scikit-learn.\n",
        "\n",
        "* `main.py`: Depends only on abstractions, not concrete libraries.\n",
        "\n",
        "* Makes it easy to swap out `SklearnRFModel` with `XGBoostModel` or even a deep learning model.\n",
        "\n",
        "---\n",
        "\n",
        "### Why it’s so important\n",
        "\n",
        "* Promotes flexibility: Swap out components without changing core logic.\n",
        "\n",
        "* Enables mocking and unit testing: You can fake IDataLoader during tests.\n",
        "\n",
        "* Decouples your ML pipeline from vendor lock-in (Scikit-learn, TensorFlow, etc.).\n",
        "\n",
        "* Production-ready design pattern for building ML SDKs or APIs.\n",
        "\n",
        "---\n",
        "\n",
        "### UML Class Diagram\n",
        "\n",
        "| ![space-1.jpg](https://github.com/Tanu-N-Prabhu/Python/blob/master/Img/uml_dip.png?raw=true) |\n",
        "|:--:|\n",
        "|Designed by Author|\n",
        "\n",
        "####  UML Class Diagram Explanation\n",
        "\n",
        "1. `IDataLoader` (Abstract Class / Interface)\n",
        "    * This is an abstraction.\n",
        "    * It declares a method `load_data()`.\n",
        "    * Any data loader class (e.g., Scikit-learn, CSV, API-based) must implement this method.\n",
        "\n",
        "2. `IModel` (Abstract Class / Interface)\n",
        "    * Another abstraction defining two essential ML behaviors:\n",
        "        * `train(X, y)`\n",
        "        * `evaluate(X, y)`\n",
        "    * Different model implementations (Random Forest, XGBoost, etc.) adhere to this interface.\n",
        "\n",
        "3. `SklearnDataLoader` (Concrete Class)\n",
        "    * Implements `IDataLoader`.\n",
        "    * Loads data using Scikit-learn (in this example, the Iris dataset).\n",
        "    * Fully replaceable with other loaders (e.g., `PandasCSVLoader`, `SQLDataLoader`) without changing the rest of the code.\n",
        "\n",
        "4. `SklearnRFModel` (Concrete Class)\n",
        "    * Implements `IModel`.\n",
        "    * Uses a `RandomForestClassifier` from Scikit-learn internally.\n",
        "    * Can be swapped out with any model implementing `IModel`(e.g., `XGBoostModel`, `KerasModel`).\n",
        "\n",
        "5. `main.py`\n",
        "    * Acts as the high-level module.\n",
        "    * It depends on the interfaces `IDataLoader` and `IModel`, not on the concrete implementations.\n",
        "    * This allows complete flexibility: inject any compatible class without modifying the pipeline logic.\n",
        "\n",
        "---\n",
        "\n",
        "### How This Reflects Dependency Inversion Principle\n",
        "* Abstractions (interfaces) define contracts both high-level (`main.py`) and low-level (`SklearnDataLoader`, `SklearnRFModel`) modules rely on.\n",
        "\n",
        "* High-level logic does not care how data is loaded or how the model is implemented.\n",
        "\n",
        "* Enables inversion of control, objects are passed in (“injected”), not created inside.\n",
        "\n",
        "---\n",
        "\n",
        "### Applications\n",
        "\n",
        "* Plug-and-play AutoML frameworks.\n",
        "\n",
        "* calable ML SDKs for teams or open-source projects.\n",
        "\n",
        "* Backend ML APIs where models can be swapped dynamically.\n",
        "\n",
        "* Systems where testing, logging, or monitoring is critical.\n",
        "\n",
        "---\n",
        "\n",
        "### Conclusion\n",
        "By following the Dependency Inversion Principle, you elevate your ML projects from experimental notebooks to clean, scalable systems. It's the key to writing machine learning code that doesn't just work, it lasts. This is what separates a good ML engineer from a great software-engineering-minded one. Thanks for reading my article, let me know if you have any suggestions or similar implementations via the comment section. Until then, see you next time. Happy coding!\n",
        "\n",
        "---\n",
        "\n",
        "### Before you go\n",
        "* Be sure to Like and Connect Me\n",
        "* Follow Me : [Medium](https://medium.com/@tanunprabhu95) | [GitHub](https://github.com/Tanu-N-Prabhu) | [LinkedIn](https://ca.linkedin.com/in/tanu-nanda-prabhu-a15a091b5) | [Python Hub](https://github.com/Tanu-N-Prabhu/Python)\n",
        "* [Check out my latest articles on Programming](https://medium.com/@tanunprabhu95)\n",
        "* Check out my [GitHub](https://github.com/Tanu-N-Prabhu) for code and [Medium](https://medium.com/@tanunprabhu95) for deep dives!\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "7gFOJ8iSEWp9"
      }
    }
  ]
}