methodology/methodology-development.md

Sections

• Gitmoji Reference
• Setup (Version Control, Environments, Documentation)
• Develop (Version Control, Server Management, Documentation)
• Deploy
• Deliver?

Setup

Project Planning

• Always have some kind of roadmap
  • non-linear roadmaps
• Setup Dev Machine
• Setup Environments
  • You should be able to run the code and develop without a dedicated server. Get virtualized with docker-compose.
  • Python setup on Ubuntu Server
• Setup Version Control
  • Connect to your repos using ssh.
  • Install and use pre-commit hooks on your dev machine
  • Setup Travis-CI build testing
  • Add an .EditorConfig file
• Documentation
  • Your repo should have a one-liner for isntallation andd running: (docker-compose.yml + doco up)

Develop

Python Developement

• Python Standard Library:
  • argparse to build CLI programs
  • pathlib for file-IO
• Pyhon modules
• pipx or pyenv for environment management.
• Fabric for deployment over SSH
• Avoid code smells

Develop with Version Control

# Git Log A DOG acronym
git log --all --decorate --oneline --graph

# when you have uncommitted changes
git add --patch <pattern>

# check what you've staged
git diff --cached

# NEW DEVELOPMENT FLOW
# clone repo with local_name enables several clones
git clone <repo> <local_name>

# optional: checkout (i.e switch to) a branch to continue developing from there
git checkout <feature/featurename>

# optional if you switched branch on the last step
# switched branch: branch out from 
git checkout <feature/newfeature>
git branch <feature/newfeature>
git push --set-upstream origin <feature/newfeature>

# delete branch locally (-d: normal, -D: force)
git branch -D <branch_name_1> <branch_name_2> <branch_name_1>
# delete remote
git push origin -d <branch_name>

# Comparing / Merging two branches
git merge --no-commit --no-ff <feature/otherfeaturebranch>

# get file from another branch
git checkout <otherbranch> -- path/to/dir

# delete branches not present on remote
git remote prune origin

# see all commits that affected somefile
git log --follow -- filename

# search code in all history
git log -S "some_old_varname" --oneline --all

Submitting multiple PRs

• Interactive rebase
• Rebase & Update PR

Context:

1. submit PR for feature1
2. from feature1 branch make new branch feature2
3. commit and push feature2 changes
4. submit PR
5. Both PRs approved

# there should be one or more flows that account for cases where:
# -> feature1 gets change requests
# -> feature1 is rejected and made obsolete

• Documentation
  • Use Google's Markdown Style Guide for written documentation.
  • Conceptual graphical documentation as inline embedded images.
  • Save Matplotlib / Seaborn plots as .svg.
  • Sketches and handwritten illustrations scanned / photographed as .png/.jpg.
  • Scanned / photographed illustrations should be cropped, contrast and alligned.
• DevOps
  • Do CodeReviews
  • Use scrum methodology
  • Test and Production environments must be the same and very well defined: FreeBSD image or Ubunut Docker container
• UX/UI
  • Understand, Ideate and Test, Implement UX.
  • Avoid dark patterns
• Visualization
  • problem: it's difficult to include visualization in this workflow but essential to fast prototyping.
  • Perhaps text-to-diagram
  • AsciiFlow
  • Spectral, a diverging colormap from matplotlib.
  • Tikz Editor
  • Matplotlib book of examples
• API
  • Requesting - use Python-requests
• Database / DataWarehouse
  • PostgreSQL (datetime functs)
  • Consider SQL (but don't use select *)
  • Be aware of feature casualties of large databases
  • Data Science - Hierarchy of Needs
• Security
  • Audit security
  • Understand SSH tunnels
  • Anticipatory Failure Determination
  • Secret Management for APIs
• Programming Personalizations
  • Code-tags (PEP-0350)
  • vscode tip: turn off openDiffOnClick
  • git bash solarized 🌞
  • Music for programming
  • OpenStack: Linting and Pre-commit Hooks
• Deploying
  • Virtualize Everything! Awesome-Compose.
• AI / ML / Modelling
  • open source cousin to GPL3
• Philosophy / Mindset / Principles
  • the-importance-of-humility-in-software-development
  • doing it right vs doing it on time
  • Closed Source Software: If you cannot check what it does / how it works, do not use it for the sake of security.
  • Software Design X Dieter Rams
• Collaboration / Working in a team
  • SPACE (satification, performance, activity, communication, efficiency)
  • Google's Four DevOp KPIs
  • Details about pair programming
  • Is extreme programming a thing?
  • Collaboration (tools: csvbox, pyodide)
  • Resolve Joel's 12 Questions
  • keeping the repo nice
• Misc
  • Things you should do now
  • Follow naming conventions
  • Follow software engineering laws
  • There's some gold nuggets this reddit post
• Mac Software
  • Lulu Firewall
  • NightOwal