Updating RobotPy source code to match WPILib

Every year, the WPILib team makes improvements to WPILib, so RobotPy needs to be updated to maintain compatibility. While this is largely a manual process, we now use a tool called git-source-track to assist with this process.

Note

git-source-track only works on Linux/OSX at this time. If you’re interested in helping with the porting process and you use Windows, file a github issue and we’ll try to help you out.

Using git-source-track

First, you need to checkout the git repo for allwpilib and the RobotPy WPILib next to each other in the same directory like so:

allwpilib/
robotpy-wpilib/

The way git-source-track works is it looks for a comment in the header of each tracked file that looks like this:

# validated: 2015-12-24 DS 6d854af athena/java/edu/wpi/first/wpilibj/Compressor.java

This stores when the file was validated to match the original source, initials of the person that did the validation, what commit it was validated against, and the path to the original source file.

Finding differences

From the robotpy-wpilib directory, you can run git source-track and it will output all of the configured files and their status. The status codes include:

  • OK: File is up to date, no changes required

  • OLD: The tracked file has been updated, `git source-track diff FILENAME can be used to show all of the git log messages and associated diffs.

  • ERR: The tracked file has moved or has been deleted

  • --: The file is not currently being tracked

Sometimes, commits are added to WPILib which only change comments, formatting, or mass file renames – these don’t change the semantic content of the file, so we can ignore those commits. When identified, those commits should be added to devtools/exclude_commits.

Looking at differences

Once you’ve identified a file that needs to be updated, then you can run:

git source-track diff FILENAME

This will output a verbose git log command that will show associated commit messages and the diff output associated with that commit for that specific file. Note that it will only show the change for that specific file, it will not show changes for other files (use git log -p COMMITHASH in the original source directory if you want to see other changes).

After running git source-track diff it will ask you if you want to validate the file. If no python-significant changes have been made, then you can answer ‘y’ and the validation header will be updated.

Adding new files

Unfortunately, git-source-track doesn’t currently have a mechanism that allows it to identify new files that need to be ported. We need to do that manually.

Dealing with RobotPy-specific files

We don’t need to track those files; git source-track set-notrack FILENAME takes care of it.

After you finish porting the changes

Once you’ve finished making the appropriate changes to the python code, then you should update the validation header in the source file. Thankfully, there’s a command to do this:

git source-track set-valid FILENAME

It will store the current date and the tracked git commit.

Additionally, if you answer ‘y’ after running git source-track diff FILENAME, then it will update the validation header in the file.

HAL Changes

If there are changes to the HAL, we have some scripts that should be able to help out here.

  • devtools/hal_fix.sh: This detects errors in the HAL, and if you pass it the --stubs argument it can print out the correct HAL definitions or a python stub. Use --help for more information.

Syntax/Style Guide

Except where it makes sense, developers should try to retain the structure and naming conventions that the Java implementation of WPILib follows. There are a few guidelines that can be helpful when translating Java to Python:

  • Member variables such as m_foo should be converted to self.foo

  • Private/protected functions (but NOT variables) should start with an underscore

  • Always retain original javadoc documentation, and convert it to the appropriate standard python docstring (see below)

Converting javadocs to docstrings

There is an HTML page in devtools called convert_javadoc.html that you can use. The way it works is you copy a Java docstring in the top box (you can also paste in a function definition too) and it will output a python docstring in the bottom box. When adding new APIs that have documentation, this tool is invaluable and will save you a ton of time – but feel free to improve it!

Enums

Though Python 3 does have an enum module, we currently are not using it to implement the enums found in the Java WPILib (this may change in the future). Instead, we define a class that has static variables with appropriate names and assign integers appropriately:

class SomeObject:

    class MyEnum:
        VALUE1 = 1
        VALUE2 = 2

Many WPILib classes define various enums, see existing code for example translations.

Synchronized

The python language has no equivalent to the Java synchronized keyword. Instead, create a threading.RLock instance object called self.lock, and surround the internal function body with a with self.lock: block:

def someSynchronizedFunction(self):
    with self.lock:
        # do something here...

Interfaces

While we define the various interfaces for documentation’s sake, the Python WPILib does not actually utilize most of the interfaces.

Final thoughts

Before translating WPILib Java code to RobotPy’s WPILib, first take some time and read through the existing RobotPy code to get a feel for the style of the code. Try to keep it pythonic and yet true to the original spirit of the code. Style does matter, as students will be reading through this code and it will potentially influence their decisions in the future.

Remember, all contributions are welcome, no matter how big or small!