DannyDannyDanny/Methodology

Repo Roadmap

• 2021Q1: Gitmoji -> (general) Methodology ✅
• 2021Q2: Version control methodology
  • Oneliner build / install / run / deploy
  • Git (Branching, Merging and Pull Requests)
  • Default environment section
• 2021Q3: Decide on sections
  • Organize other methodology into hierarchy
  • Split hierarchy into paragraphs / sections
• 2021Q4
  • Add sub-pages
  • Add Custom utils.

This methodology repo is focused on Python Data Processing based projects with som external dependencies (APIs or DBs). The repo is actively being updated; Leave a start bookmark it or Watch the repo to get notificatied of updates.

Version control

Commit messages and gitmoji

Use the format :gitmoji: <commit message> in your commit messages. Only proper nouns, CamelCase and class names should have uppercases letters. Avoid using multiple gitmojis.

emoji	code	description
✨	:sparkles:	add new feature / function
🎨	:art:	refactor code / improve readibility / structure and lint flake8
🥅	:goal_net:	try/raise/handle errors or warnings
⚡	:zap:	improve performance / coverage / profiling
🐛	:bug:	fix this bug
🚚	:truck:	moving / renaming
♻️	:recycle:	work related to CI/CD, webhooks, automated tests
🚧 / 💥	:construction:/:boom:	work in progress / breaking changes
📝	:memo:	doc / logging / verbosity / code comments
🔥 / ⚰️	:fire:/:coffin:	remove (dead) code / file
👽	:alien:	changes related to APIs and Databases
📊	:bar_chart:	clean / transform / validate data
💄	:lipstick:	improve UI / UX, CLI or visualizations
📌	:pushpin:	dependency and environment versioning
🙈	:see_no_evil:	changes to .gitignore and exclusively local files
🔍	:mag:	changes in connection with PR-review change-requests

Missing a gitmoji? Get inspiration. Submit an issue (or Pull Request).

Oneliner install and run

When using git repositories, connect with ssh.

The README for your project should contain a one-liner code that dowloads, installs and runs a mini-test.

git clone https://github.com/user/repo.git cool-repo && cd cool-repo
pipenv install
pipenv run python .
#cd .. && rm -rf cool-user-repo-online

Git Flows

In progress! 🚧 Use git hooks!

git log --all --decorate --oneline --graph
# Git Log A DOG acronym

New Development Flow

# clone repo with local_name enables several clones
git clone <repo> <local_name>

# optional: checkout (i.e switch to) a branch to continue developing from there
git checkout <feature/featurename>

# optional if you switched branch on the last step
# switched branch: branch out from 
git checkout <feature/newfeature>
git branch <feature/newfeature>
git push --set-upstream origin <feature/newfeature>

Comparing / Merging two branches

git merge --no-commit --no-ff <feature/otherfeaturebranch>

Submitting multiple PRs

• Interactive rebase
• Rebase & Update PR

Context:

1. submit PR for feature1
2. from feature1 branch make new branch feature2
3. commit and push feature2 changes
4. submit PR
5. Both PRs approved

# there should be one or more flows that account for cases where:
# -> feature1 gets change requests
# -> feature1 is rejected and made obsolete

Repo Badges

Coming soon! 🚧

Documentation

• Use markdown with Google Style Guide for written documentation.
• Conceptual graphical documentation as inline embedded images.
• Save Matplotlib / Seaborn plots as .svg.
• Sketches and handwritten illustrations scanned / photographed as .png/.jpg.
• Scanned / photographed illustrations should be cropped, contrast and alligned.

Cons of tables	Pros of tables
Great splitting information	Painful to format in MD

Python

Python is notoriously painful to manage. Use pipx or pyenv.

Pathlib

from pathlib import Path
Path('.').iterdir()
Path('.').absolute()

Deploying to Servers

When deploying to FreeBSD and Linux servers, place the code in the /home directory.

Other methdology

• Closed Source Software: If you cannot check what it does / how it works, do not use it for the sake of security.
• Things you should do now
• Follow naming conventions
• Follow software engineering laws
• Use scrum methodology
• Anticipatory Failure Determination
• Understand, Ideate and Test, Implement UX.
• Do CodeReviews
• Use databases
  • Consider SQL (but don't use select *)
  • Be aware of feature casualties of large databases
• Data Science - Hierarchy of Needs
• Avoid dark patterns
• Security
  • Audit security
  • Understand SSH tunnels
• Use non-linear roadmaps
• Programming Personalizations
  • git bash solarized 🌞
• Test and Production environments must be the same and very well defined: FreeBSD image or Ubunut Docker container
• Music for programming
• Resolve Joel's 12 Questions
• Visualization problem: it's difficult to include visualization in this workflow but essential to fast prototyping.
  • Perhaps text-to-diagram
  • AsciiFlow
• There's some gold nuggets this reddit post
• the-importance-of-humility-in-software-development
• OpenStack: Linting and Pre-commit Hooks
• Details about pair programming
• Is extreme programming a thing?
• Collaboration (tools: csvbox, pyodide)
• Virtualize Everything! Awesome-Compose.
• open source cousin to GPL3
• Divide year into 26 fortnights labelled A-Z

Project Portfolio

• Consumption Predictor