Spinup a new python project, manually
This is a recipe for how I spin up a new python project.
github
Starting from github, I create a new project. Choose to include readme
, .gitignore
(Python), and a license.
Which license to choose
Generally, I will choose MIT
.
ref: (https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository)
ref: (https://gist.github.com/nicolasdao/a7adda51f2f185e8d2700e1573d8a633)
ref: (https://opensource.guide/legal)
Directory structure
This is the structure I will use:
README.md
LICENSE
widget/\__init.py__.py
widget/widget.py
widget/other.py
tests/test_widget.py
requirements.txt
requirements-dev.txt
.gitignore
.github
Makefile
module
I come from a world where there is a ./src
and ./src/test
directory. I I understand that this is frowned upon in python world. Oh well. I like to seperate the code from all the other .git fluff
Given a module called widget
:
If a single file, add to root as src/widget.py
If multiple files, add a directory src/widget
along with a __init__
file
ref: Hitchhiker’s Guide to Python
tests
Place unit tests in directory src/tests
. Unit test files will have pattern test_*py
README.md
Explain the purpose, how to install it, how to consume it, and how to contribute
LICENSE
Auto-generated from github. If you did not let github generate one for you, add one.
.gitignore
Auto-generated from github.
git actions
git actions will leave in .git
Create module
I will list two approaches:
- script
- library
Script
Assuming that this will run as a script with a main
method:
Create your module file widget.py
with the following code:
def main():
print("Hello World!")
if __name__ == "__main__":
main()
After creating this, validate:
python3 widget.py
Library
TODO
Dependencies
I am still adjusting to the right way to handle dependencies with python. I love the simplicity of NPM, and pyenv
seems to be a great similiar approach. But I have struggled with it in github actions. So for now, I will use:
pip
for installing dependencies
requirements.txt
for run-time dependencies
requirements-dev.txt
for development dependencies
Run-time Dependencies
Create requirements.txt
and add to it as needed
Dev Dependencies
I am currently using:
- yapf: formatting (this has several competitors)
- pylint: style (this has several competitors)
- pytest: unit testing (I want to find or make something better, I am spoiled by tools in node/JS and java)
Create requirements-dev.txt
with the following entries (github will help us bring these to most current versions):
yapf == 0.40.0
pylint == 3.0.0
pytest == 7.4.0
Build
Create a Makefile
with the following entries:
all: default
clean:
deps:
pip install -r requirements.txt
dev_deps:
pip install -r requirements-dev.txt
check-format: dev_deps
yapf -rd src/
format: dev_deps
yapf -ri src/
lint: check-format
pylint -r n src/
lint-no-error:
pylint --exit-zero -r n src/
test: init dev_deps
python3 -m pytest -v
init: clean deps
Unit Test
Create a stubbed empty unit test:
File: src/tests/test_placeholder.py
import unittest
class TestPlaceholder(unittest.TestCase):
def test_sample(self):
self.assertEqual(0, 0)
Execute make test
to verify:
> make test
pip install -r requirements.txt
pip install -r requirements-dev.txt
Requirement already satisfied: yapf==0.40.0 in /opt/homebrew/lib/python3.11/site-packages (from -r requirements-dev.txt (line 1)) (0.40.0)
Requirement already satisfied: pylint==3.0.0 in /opt/homebrew/lib/python3.11/site-packages (from -r requirements-dev.txt (line 2)) (3.0.0)
Requirement already satisfied: pytest==7.4.0 in /opt/homebrew/lib/python3.11/site-packages (from -r requirements-dev.txt (line 3)) (7.4.0)
Requirement already satisfied: importlib-metadata>=6.6.0 in /opt/homebrew/lib/python3.11/site-packages (from yapf==0.40.0->-r requirements-dev.txt (line 1)) (6.8.0)
Requirement already satisfied: platformdirs>=3.5.1 in /opt/homebrew/lib/python3.11/site-packages (from yapf==0.40.0->-r requirements-dev.txt (line 1)) (3.11.0)
Requirement already satisfied: tomli>=2.0.1 in /opt/homebrew/lib/python3.11/site-packages (from yapf==0.40.0->-r requirements-dev.txt (line 1)) (2.0.1)
Requirement already satisfied: astroid<=3.1.0-dev0,>=3.0.0 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (3.0.1)
Requirement already satisfied: isort<6,>=4.2.5 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (5.12.0)
Requirement already satisfied: mccabe<0.8,>=0.6 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (0.7.0)
Requirement already satisfied: tomlkit>=0.10.1 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (0.12.1)
Requirement already satisfied: dill>=0.3.6 in /opt/homebrew/lib/python3.11/site-packages (from pylint==3.0.0->-r requirements-dev.txt (line 2)) (0.3.7)
Requirement already satisfied: iniconfig in /opt/homebrew/lib/python3.11/site-packages (from pytest==7.4.0->-r requirements-dev.txt (line 3)) (2.0.0)
Requirement already satisfied: packaging in /opt/homebrew/lib/python3.11/site-packages (from pytest==7.4.0->-r requirements-dev.txt (line 3)) (23.2)
Requirement already satisfied: pluggy<2.0,>=0.12 in /opt/homebrew/lib/python3.11/site-packages (from pytest==7.4.0->-r requirements-dev.txt (line 3)) (1.3.0)
Requirement already satisfied: zipp>=0.5 in /opt/homebrew/lib/python3.11/site-packages (from importlib-metadata>=6.6.0->yapf==0.40.0->-r requirements-dev.txt (line 1)) (3.17.0)
python3 -m pytest -v
=============================================================================================================== test session starts ================================================================================================================
platform darwin -- Python 3.11.6, pytest-7.4.0, pluggy-1.3.0 -- /opt/homebrew/opt/python@3.11/bin/python3.11
cachedir: .pytest_cache
rootdir: xxx
collected 1 item
tests/test_placeholder.py::TestPlaceholder::test_sample PASSED [100%]
================================================================================================================ 1 passed in 0.00s =================================================================================================================
Github Actions
You can create a github action to test the project upon push. This is can be done from github or add the following file:
.github/workflows/python-package.yml
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Python package
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v3
- name: Set up Python $
uses: actions/setup-python@v3
with:
python-version: $
- name: Upgrade pip
run: |
pip install --upgrade pip
- name: Install dependencies
run: |
make init
- name: Run unit tests
run: |
make test
- name: Check Format
run: |
make check-format
- name: Check Style
run: |
# make lint-no-error
make lint
Commit and push this to github, go to actions to see in action. Pun intended.
Add build badge
Navigate to the action on github, click on the action (Python package
), click the ...
on right side, and create status badge. Copy status badge for markdown.
Edit the readme in root directory, and paste in this badge.
Format / Lint / Pytest config
Need to get common one of these but for now, pull these from here:
curl https://raw.githubusercontent.com/jasonray/RPN-calculator-python/main/.pytest.ini > .pytest.ini
curl https://raw.githubusercontent.com/jasonray/RPN-calculator-python/main/.pylintrc > .pylintrc
curl https://raw.githubusercontent.com/jasonray/RPN-calculator-python/main/.style.yapf > .style.yapf
Enable dependabot
Create .github/dependabot.yml
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: "pip" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "daily"
After first run, this will likely create a set of PRs in github.
Enable Branch Protection
Settings | Branches | Add | main |
Require a pull request before merging
(no approvals for solo projects)
Require status checks to pass before merging