Spinup a new python project, manually

November 10, 2023 5 minute read

This is a recipe for how I spin up a new python project.

github

Starting from github, I create a new project. Choose to include readme, .gitignore (Python), and a license.

Which license to choose

Generally, I will choose MIT.

ref: (https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository)
ref: (https://gist.github.com/nicolasdao/a7adda51f2f185e8d2700e1573d8a633)
ref: (https://opensource.guide/legal)

Directory structure

This is the structure I will use:

README.md
LICENSE
widget/\__init.py__.py
widget/widget.py
widget/other.py
tests/test_widget.py
requirements.txt
requirements-dev.txt
.gitignore
.github
Makefile

module

I come from a world where there is a ./src and ./src/test directory. I I understand that this is frowned upon in python world. Oh well. I like to seperate the code from all the other .git fluff

Given a module called widget:
If a single file, add to root as src/widget.py
If multiple files, add a directory src/widget along with a __init__ file

ref: Hitchhiker’s Guide to Python

tests

Place unit tests in directory src/tests. Unit test files will have pattern test_*py

README.md

Explain the purpose, how to install it, how to consume it, and how to contribute

LICENSE

Auto-generated from github. If you did not let github generate one for you, add one.

.gitignore

Auto-generated from github.

ref: python
ref: MacOS

git actions

git actions will leave in .git

Create module

I will list two approaches:

script
library

Script

Assuming that this will run as a script with a main method:

Create your module file widget.py with the following code:

def main():
    print("Hello World!")

if __name__ == "__main__":
    main()

After creating this, validate:

python3 widget.py

Library

TODO

Dependencies

I am still adjusting to the right way to handle dependencies with python. I love the simplicity of NPM, and pyenv seems to be a great similiar approach. But I have struggled with it in github actions. So for now, I will use:

pip for installing dependencies requirements.txt for run-time dependencies requirements-dev.txt for development dependencies

Run-time Dependencies

Create requirements.txt and add to it as needed

Dev Dependencies

I am currently using:

yapf: formatting (this has several competitors)
pylint: style (this has several competitors)
pytest: unit testing (I want to find or make something better, I am spoiled by tools in node/JS and java)

Create requirements-dev.txt with the following entries (github will help us bring these to most current versions):

yapf == 0.40.0
pylint == 3.0.0
pytest == 7.4.0

Build

Create a Makefile with the following entries:

all: default

clean: 

deps:
	pip install -r requirements.txt

dev_deps:
	pip install -r requirements-dev.txt

check-format: dev_deps
	yapf -rd src/

format: dev_deps
	yapf -ri src/

lint: check-format
	pylint -r n src/

lint-no-error: 
	pylint --exit-zero -r n src/

test: init dev_deps
	python3 -m pytest -v

init: clean deps

Unit Test

Create a stubbed empty unit test:

File: src/tests/test_placeholder.py

import unittest

class TestPlaceholder(unittest.TestCase):

    def test_sample(self):
        self.assertEqual(0, 0)

Execute make test to verify:

> make test
pip install -r requirements.txt
pip install -r requirements-dev.txt
Requirement already satisfied: yapf==0.40.0 in /opt/homebrew/lib/python3.11/site-packages (from -r requirements-dev.txt (line 1)) (0.40.0)
Requirement already satisfied: pylint==3.0.0 in /opt/homebrew/lib/python3.11/site-packages (from -r requirements-dev.txt (line 2)) (3.0.0)
Requirement already satisfied: pytest==7.4.0 in /opt/homebrew/lib/python3.11/site-packages (from -r requirements-dev.txt (line 3)) (7.4.0)
Requirement already satisfied: importlib-metadata>=6.6.0 in /opt/homebrew/lib/python3.11/site-packages (from yapf==0.40.0->-r requirements-dev.txt (line 1)) (6.8.0)
Requirement already satisfied: platformdirs>=3.5.1 in /opt/homebrew/lib/python3.11/site-packages (from yapf==0.40.0->-r requirements-dev.txt (line 1)) (3.11.0)
Requirement already satisfied: tomli>=2.0.1 in /opt/homebrew/lib/python3.11/site-packages (from yapf==0.40.0->-r requirements-dev.txt (line 1)) (2.0.1)
Requirement already satisfied: astroid<=3.1.0-dev0,>=3.0.0 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (3.0.1)
Requirement already satisfied: isort<6,>=4.2.5 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (5.12.0)
Requirement already satisfied: mccabe<0.8,>=0.6 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (0.7.0)
Requirement already satisfied: tomlkit>=0.10.1 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (0.12.1)
Requirement already satisfied: dill>=0.3.6 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (0.3.7)
Requirement already satisfied: iniconfig in /opt/homebrew/lib/python3.11/site-packages (from pytest==7.4.0->-r requirements-dev.txt (line 3)) (2.0.0)
Requirement already satisfied: packaging in /opt/homebrew/lib/python3.11/site-packages (from pytest==7.4.0->-r requirements-dev.txt (line 3)) (23.2)
Requirement already satisfied: pluggy<2.0,>=0.12 in /opt/homebrew/lib/python3.11/site-packages (from pytest==7.4.0->-r requirements-dev.txt (line 3)) (1.3.0)
Requirement already satisfied: zipp>=0.5 in /opt/homebrew/lib/python3.11/site-packages (from importlib-metadata>=6.6.0->yapf==0.40.0->-r requirements-dev.txt (line 1)) (3.17.0)
python3 -m pytest -v
=============================================================================================================== test session starts ================================================================================================================
platform darwin -- Python 3.11.6, pytest-7.4.0, pluggy-1.3.0 -- /opt/homebrew/opt/python@3.11/bin/python3.11
cachedir: .pytest_cache
rootdir: xxx
collected 1 item                                                                                                                                                                                                                                   

tests/test_placeholder.py::TestPlaceholder::test_sample PASSED                                                                                                                                                                               [100%]

================================================================================================================ 1 passed in 0.00s =================================================================================================================

Github Actions

You can create a github action to test the project upon push. This is can be done from github or add the following file:

.github/workflows/python-package.yml

# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python package

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

jobs:
  build:

    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.9", "3.10", "3.11", "3.12"]

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python $
      uses: actions/setup-python@v3
      with:
        python-version: $
    - name: Upgrade pip
      run: |
        pip install --upgrade pip
    - name: Install dependencies
      run: |
        make init
    - name: Run unit tests
      run: |
        make test
    - name: Check Format 
      run: |
        make check-format
    - name: Check Style 
      run: |
        # make lint-no-error
        make lint

Commit and push this to github, go to actions to see in action. Pun intended.

Add build badge

Navigate to the action on github, click on the action (Python package), click the ... on right side, and create status badge. Copy status badge for markdown.

Edit the readme in root directory, and paste in this badge.

Format / Lint / Pytest config

Need to get common one of these but for now, pull these from here:

curl https://raw.githubusercontent.com/jasonray/RPN-calculator-python/main/.pytest.ini > .pytest.ini
curl https://raw.githubusercontent.com/jasonray/RPN-calculator-python/main/.pylintrc > .pylintrc
curl https://raw.githubusercontent.com/jasonray/RPN-calculator-python/main/.style.yapf > .style.yapf

Enable dependabot

Create .github/dependabot.yml

# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
  - package-ecosystem: "pip" # See documentation for possible values
    directory: "/" # Location of package manifests
    schedule:
      interval: "daily"

After first run, this will likely create a set of PRs in github.

Enable Branch Protection

Settings

Branches

Add

main

Require a pull request before merging (no approvals for solo projects) Require status checks to pass before merging

Mighty Pulpo