This page documents the coding standards, tools and procedure that the lab applies to its python projects.

Coordinator: Javier Peralta

Contributors: Ariel Mora, Daniel García Vaglio

Note: Most of these sections (specially the tools ones) are meant as a small show case and generally have link to the relevant documentation. Please read this links since the information presented here is not exhaustive nor complete.

Python projects should strictly follow PEP8.

The following tools can aid you to keep your styling on point:

Flake8 is a code checker, it will check python files and raise errors and warnings when code breaks PEP8 or if there is a syntax error.

To install:

apt-get install flake8

Usage:

flake8 path/to/code/

Yet another python formatter (yapf) is a code formatter, a program that reformat code to comply with a style guide (like PEP8 which is the default style). Running this over your python project will clear most PEP8 compliance errors.

To install:

apt-get install yapf3

Usage:

yapf3 -ir path/to/code_dir

or

yapf3 -i path/to/code/file.py

Logging is a very important part of every software project, because it is the best way to deliver valuable information to our users. Logs are used to report errors, warnings, or just general information of how the program is executing. Good logs help us to discover why the program fails some times, they help us diagnose problems with our programs and help us plan strategies for improvement. It is crucial to maintain a good and healthy logging practices so that you can increase the maintainability of your code.

Python developers, knowing that logs are so important, have developed a module just for this: the Logging module. There are 5 default levels of logging, ordered from most important to less important are: CRITICAL, Error, Warning, Info, Debug.

CRITICAL logs are messages that are shown to the user that report that something very bad happened. The program is not able to continue its execution after that failure and it may cause colateral damage (affect a database, a network, files…).

Error logs are used when something goes wrong, but only a part is affected. Most failures can be logged as Error.

The Warning level is used to report errors from which your application is able to recover autonomously. Your system won't crash because of a Warning, but you are alerting the user that something is not OK. Warning logs can also be used in libraries when you want to tell your users that there are going to be some API changes. For example, if you want to change the name of a function, or if a method will be deprecated, the best thing to do is to create a Warning to announce that before doing the changes.

Info logs are for general information, there is nothing going wrong with your program, but you want to inform the user about events that might be of interest. Messages like “The file has been opened”, “Connection established” are common Info messages.

Finally Debug logs are used only during development. Let's say that you are not sure why your program is behaving in a certain way, and you believe that there might be a problem with the value of a certain variable. So you can create a Debug log to print that variable's value, to see if there really is a problem.

When you define a logging level, you are telling Python which messages you want to receive. If you define a level A, you will receive all A messages and all messages with more priority. If you say that you want a Error level, so you will receive Error and CRITICAL logs only. If you define Debug level, you will receive all logs. For most cases you will want to set Info or Warning levels.

The recommended project layout for a python projected named

ptoject_name

is as follows:

project_name
├── AUTHORS.rst
├── docs
├── examples
│  └── example1.py
├── LICENSE
├── Makefile
├── MANIFEST.in
├── README.rst
├── requirements.dev.txt
├── requirements.txt
├── setup.cfg
├── setup.py
├── src
│  └── package_name
│     ├── __init__.py
│     └── module1.py
├── tests
└── tox.ini

cookiecutter is a command-line utility that creates projects from cookiecutters (project templates). E.g. Python package projects, jQuery plugin projects.

To install:

apt-get install cookiecutter

Arcos template for python is hosted in this repo, to create a new project run:

cookiecutter https://github.com/arcoslab/arcos-python-cookiecutter

after the initial clone you can other new repos refering to the cookicutter by name:

cookiecutter arcos-python-cookiecutter

The lab have some Python 2 repos that need to migrated to Python3. The process will be as follow:

1. Migrate all repos one by one to support both python 2 and 3 from the same file
2. After all repos are migrated we will drop python 2 support for good

For each repo the following process will be applied:

1. An issue to track the repo's migration progress will be opened
2. Open a new py3stage1 branch
3. Using futurize apply stage1 changes to all files
4. Open a pull request and have another member check the changes.
5. Tag the pre-stage1 release so people can easily use that if an issue raises
6. Merge the changes to master, rebase all other branches from master and resolve merge conflicts
7. Test the changes, this is probably going to be manual since most repos have no unit tests. This may be a good time to write some.
8. Open a new py3stage2 branch.
9. Repeat the process for stage1, testing both for python2 and python3
10. Check everything twice
11. Close the issue

Any problems encountered due to migration should be reported with a GitLab issue on the corresponding repo.

To see progress please take a look to the following project tracker on github