您如何喜欢Python中的此依赖项管理选项?

我最近决定现在是时候在我的Python项目中理清依赖管理的主题,并开始寻找一种完全适合我的解决方案。我尝试了Pipenv,研究了诗歌文档,阅读了其他相关文章。不幸的是,我从未找到完美的解决方案。结果,我发明了一种新的自行车,我的方法,我建议在削减的情况下进行讨论。


问题


但是,在直接描述该方法之前,我想解释为什么会出现这种需求以及为什么现有的解决方案不适合我。


作为我工作的一部分,我经常将Python用于两个目的:使用Jupyter笔记本进行数据分析和机器学习,或者以某种方式准备数据的小型Python脚本。我想指出的是,我没有参与程序包的创建及其在Pypi中的发布(因此,在本文中,我不关注此过程)。


我经常需要运行很久以前创建的脚本或记事本。因此,我需要以某种方式修复依赖项版本并在虚拟环境中运行脚本和记事本。另一方面,有时在新版本的库中,可能会出现一些功能,这些功能将改善旧记事本或脚本的结果。例如,在scikit-learn(机器学习的库)中,他们可以添加新算法的实现,该算法对我的情况非常有用。


很多时候,在开发脚本时,我还必须安装一些仅开发所需的其他依赖项。例如,由于我使用VSCode进行开发,因此需要在虚拟环境中安装pylint。与我一起工作的其他人可以使用完全不需要这种依赖的其他开发工具。


基于这些假设,我对依赖项管理提出了以下要求:


  1. 我应该能够将主要的依赖关系(需要运行脚本)和仅用于开发的依赖关系分开。
  2. 我想指定中央依赖性而不参考该库的特定版本。这样,我可以轻松地更新依赖关系。另一方面,我应该能够提交库版本,以便完全重复我的虚拟环境。
  3. 对于脚本和记事本,相同的方法应该同样有效。

, ( pip freeze > requirements.txt) requirements.txt, . -, . -, requirements.txt , .


, pipenv (, , , ). , , . , , . , , . Poetry .



, , (k)Ubuntu 18.04. , . , , .


, . -, . : requirements.txt , requirements-dev.txt . , bash pip-install.


pip-install
function pip-install() {
    packages=()
    dev_dependency=0
    requirements_file=
    while [ $# -gt 0 ]
    do
        case "$1" in
            -h|--help)
                echo "Usage: pip-install [-d|--dev] [-r|--req <file>] <package1> <package2> ..." 1>&2
                echo ""
                echo "This function installs provided Python packages using pip"
                echo "and adds this dependency to the file listing requirements."
                echo "The name of the package is added to the file without"
                echo "concreate version only if it is absent there." 
                echo ""
                echo "-h|--help        - prints this message and exits."
                echo "-d|--dev         - if the dependency is development."
                echo "-r|--req <file>  - in which file write the dependency."
                echo "    If the filename is not provided by default the function"
                echo "    writes this information to requirements.txt or to"
                echo "    requirements-dev.txt if -d parameter is provided."
                echo "<package1> <package2> ..."
                return 0
                ;;
            -d|--dev)
                shift
                dev_dependency=1
                ;;
            -r|--req)
                shift
                requirements_file="$1"
                echo "Requirements file specified: $requirements_file"
                shift
                ;;
            *)
                packages+=( "$1" )
                echo "$1"
                shift
                ;;
        esac
    done

    if ! [ -x "$(command -v pip)" ]; then
        echo "Cannot find pip tool. Aborting!"
        exit 1
    fi

    echo "Requirements file: $requirements_file"
    echo "Development dependencies: $dev_dependency"
    echo "Packages: ${packages[@]}"

    if [ -z "$requirements_file" ]; then
        if [ $dev_dependency -eq 0 ]; then
            requirements_file="requirements.txt"
        else
            requirements_file="requirements-dev.txt"
        fi
    fi

    for p in "${packages[@]}"
    do
        echo "Installing package: $p"
        pip install $p
        if [ $? -eq 0 ]; then
            echo "Package installed successfully"
            echo "$p" >> $requirements_file
            if [ $(grep -Ec "^$p([~=!<>]|$)" "$requirements_file") -eq 0 ]; then
                echo "$p" >> $requirements_file
            else
                echo "Package $p is already in $requirements_file"
            fi
        else
            echo "Cannot install package: $p"
        fi
    done
}

, : pip-install scikit-learn pip-install --dev pylint. , scikit-learn pip ( ) requirements.txt. , requirements-dev.txt. , , , ( ). , .


, pip-freeze. pip-freeze requirements.txt, requirements.lock . , pip install -r requirements.lock. pip-freeze --dev , requirements-dev.txt.


pip-freeze
function pip-freeze() {
    dump_all=0
    dev_dependency=0
    requirements_file=
    while [ $# -gt 0 ]
    do
        case "$1" in
            -h|--help)
                echo "Usage: pip-freeze [-a|--all] [-d|--dev] [-r|--req <file>]" 1>&2
                echo ""
                echo "This function freezes only the top-level dependencies listed"
                echo "in the <file> and writes the results to the <file>.lock file."
                echo "Later, the data from this file can be used to install all"
                echo "top-level dependencies." 
                echo ""
                echo "-h|--help        - prints this message and exits."
                echo "-d|--dev         - if the dependency is development."
                echo "-a|--all         - if we should freeze all dependencies"
                echo "  (not only top-level)."
                echo "-r|--req <file>  - what file to use to look for the list of"
                echo "    top-level dependencies. The results will be written to"
                echo "    the \"<file>.lock\" file." 
                echo "    If the <file> is not provided by default the function"
                echo "    uses \"requirements.txt\" or \"requirements-dev.txt\""
                echo "    if -d parameter is provided and writes the results to the"
                echo "    \"requirements.txt.lock\" or \"requirements-dev.txt.lock\""
                echo "    correspondingly."
                return 0
                ;;
            -d|--dev)
                shift
                echo "Development dependency"
                dev_dependency=1
                ;;
            -a|--all)
                shift
                dump_all=1 
                ;;
            -r|--req)
                shift
                requirements_file="$1"
                echo "Requirements file specified: $requirements_file"
                shift
                ;;
        esac
    done

    if ! [ -x "$(command -v pip)" ]; then
        echo "Cannot find pip tool. Aborting!"
        exit 1
    fi

    if [ -z "$requirements_file" ]; then
        if [ $dev_dependency -eq 0 ]; then
            requirements_file="requirements.txt"
        else
            requirements_file="requirements-dev.txt"
        fi
    fi

    lock_file="$requirements_file.lock"
    if [ $dump_all -eq 1 ] 
    then
        pip freeze > "$lock_file"
        if [ $? -eq 0 ]; then
            echo "Locked all dependencies to: $lock_file"
        else
            echo "Error happened while locking all dependencies"
        fi
    else
        cmd_output=$(pip freeze -r "$requirements_file")
        if [ $? -eq 0 ]; then
            > "$lock_file"
            while IFS= read -r line; do
                if [ "$line" = "## The following requirements were added by pip freeze:" ]; then
                    break
                fi
                echo "$line" >> "$lock_file"
            done <<< "$cmd_output"
        fi
    fi
}

, 4 : requirements.txt, requirements-dev.txt, requirements.lock requirements-dev.lock.


这两个函数的源代码存储在我的文件中(始终检查源代码!!!)。您可以将其复制到目录中~/.bash/,并使这些功能在家中可用,请在以下位置添加以下几行.bashrc


if [ -f ~/.bash/pip_functions.sh ]; then
    source ~/.bash/pip_functions.sh
fi

结论


我刚开始使用此解决方案,也许还没有发现任何缺陷。通常,此职位的构想是为了讨论此方法以及他是否享有生命权。如果您发现任何问题,我将很高兴听到他们的消息。


聚苯乙烯

, Python , ( ) .


All Articles