What I learned this month: Github Actions and pre-commit

This is yet again another attempt to reboot the dev part of this blog. I’m not successful, but at least I’m persistent 😉

Introduction

I’m kicking off a new serie of posts. Every month, I plan on writing about the new stuff that I’ve discovered in the broader field of software development. It’s an attempt to share the knowledge that I may have gained during that time, and also to show the world that you can be a somewhat experienced software developer and still be discovering new stuff every month.

I have observed that I tend to discover the latest hype about at the time it stops being cool. If you’re like me, we can make up for this latency by sharing ideas. Also, it requires quite a bit of time to play with software that isn’t yet ready for anything besides its original intended use case. So, let’s not waste our time.

I don’t plan on writing well researched articles, because aiming too high is a sure way for me to fail at building a habit, and that’s one of the main goals here. I’ll probably be a bit terse, with only the main links and my impressions. I hope you’ll understand. (and if you don’t, I’m not making you read this 😉 )

The two tools that I’ve selected for this month’s article are Github Actions and pre-commit.

Github Actions

D’oh, you might say. It’s been around for a while but I didn’t have a chance to use it yet, because in Fedora we run our CI on a Jenkins instance that the good CentOS folks provide us with.

But for some projects, it does not make much sense to run our unit tests on specific Fedora versions, and it’s always good to have an alternative. I’ve set up Github Actions to run CI on a couple projects that I maintain, such as fedora-messaging and flask-healthz, to test-drive it. The Python SIG in Fedora also provides a Github Action to run tox on a Fedora container, which is nice.

Another use: since Dependabot has eaten by Github, they have removed the feature to auto-merge dependency updates according to their semver classification. And I did not enjoy manually approving patch updates to my dependencies, since what I basically did was to make sure CI passed. That’s a bot’s job. So I’ve set up a Github Action that either merges patch and minor updates (but not backwards-incompatible major versions), or just approves them, waiting for Mergify to do the actual merge.

It’s nice, it’s fast, I like it. Maybe I’ll end up keeping our CentOS CI Jenkins instance only for integration tests.

pre-commit

I discovered pre-commit recently and I think it has potential. It’s a well known fact of software development that the sooner you catch a bug, the lesser that bug costs you. Find a bug during development (and even while typing the code, thanks to linters embedded in your editor) and you’ll be much better off than if you found it after production deployment.

The point of pre-commit is to run checks before the code is committed to Git. That means fast checks such as linters and formatters, and probably now your entire unit tests suite, but your mileage may vary. As for me, I usually run black, isort and flake8 as part of Visual Studio Code, so I catch issues even sooner. However, I’m not the only one to work on my projects, and for those not using advanced text editors it is a nice safety net to run the checks before committing.

I’ve converted a couple projects to pre-commit, we’ll see how it goes.

Conclusion

It’s all for this first edition. Hopefully I’ll see you next month with more findings. In the meantime, happy hacking! 🙂

Reviews are hard

It’s a vast subject, but one thing is certain: reviewing other people’s code is hard. Because good mentoring require technical and non-technical skills (such as patience).

I would like to dive directly into a specific detail of code reviews. It’s an iterative process: author submits code for review, reviewer make suggestions, author amends or pushes more code, reviewer make different or more suggestions, and so on.

In Git, « more code » takes the form or one or more commits appended to the Pull Request (or Merge Request if you use Gitlab, for simplicity I’ll just use « Pull Request » in this piece). And « amended code » means overwriting existing commits and force-pushing, which makes the old commits disappear.

As a reviewer, it can be very annoying because what I first look for in an update is whether my suggestions have been implemented or not, and how. That’s why authors are sometimes encouraged to push new commits in their Pull Requests and never overwrite existing ones. It makes the reviewer’s job way easier, because the UI can just show the new commit and they’ll know what’s changed.

But having this policy has drawbacks. When the Pull Request is merged (by fast-forward or not) in can leave awkward commits in the history, like « implement suggestions », « fix according to review », « review fixes again », etc… And merging the PR by squashing it isn’t always relevant, sometimes I do want to have several commits, because they address different parts of the problem.

How can we solve this? Well, it would be nice if I could see the difference between the current state and the last time I reviewed regardless of whether the author has amended their commit or not. And for that, I need to have a local copy of that commit. Fortunately that’s one of the things that git is very good at: you just make a local branch that tracks the PR’s branch, and when the code is changed you make another one. And then you can diff between those branches and see what changed.

OK, sounds simple enough. I have an itch, let me scratch it.

I present you git-pr-branch! A « small » utility that will create branches from Pull Requests, and do a few things with them. You’ll be able to automatically create the PR-based branches that I just explained about. You’ll also be able to display a nice listing of the branches, their associated PR, the PR status (open or closed), and the PR URL to clickety click. And since this can end up in quite a lot of branches, there’s also a sub-command to clean all that up, and delete branches whose PR is closed.

Ironically, it’s hosted on Gitlab but at the moment it only works on Github and Pagure. I’ll add Gitlab support if I end up working more with Gitlab (something tells me it’s likely to happen in the near future), but you can also send me a patch if you want it sooner.

While writing this side project, I discovered the fantastic python library attrs. It’s really awesome, I encourage you to try it out. (as always, my side projects are a good opportunity to try out new libraries or frameworks that I discover 😉 )

The python packages are on PyPI, and for the lazy Fedora and Mageia users out there I’ve made a COPR repository that you can enable on Fedora 32 and Mageia 7. Once installed, just run git-pr-branch (or git pr-branch) to discover the available commands.

Feel free to tell me what you think of the tool. Do you like the idea? Are you going to use it in your reviewer workflow? Did I bother writing code again when there is an obvious and better tool to do it? Let me know! 🙂

Source: https://bonkersworld.net/code-reviews

[EDIT] I’ve made a COPR repo, added the link.
[EDIT2] It now works with Pagure too, added the reference.

Les hommes parlent trop

Sous ce titre un peu provocateur, je voudrais vous parler d’une application toute simple pour mesurer le temps de parole. Contexte :

• Je suis informaticien, j’écris des programmes.
• Je fais partie de l’équipe d’orga des Cafés Poly à Paris, qui organisent des soirées discussion-débat autour de la polyamorie (ou « polyamour » )

Lors des cafés polys, nous avions l’impression que les hommes parlaient significativement plus que les femmes et prenaient plus souvent la parole. C’est cohérent avec les observations et les analyses féministes, donc nous n’étions pas très surpris⋅es, mais nous voulions pouvoir le mesurer pour prendre des décisions sur cette problématique.

J’avais entendu parler plus tôt de la page Are Men Talking Too Much, qui est un simple double-compteur de temps de parole (avec deux boutons). Ça aurait pu nous convenir mais nous voulions des infos supplémentaires :

• comment se répartissent les prises de parole ? Si une femme parle longtemps elle va peut-être masquer le fait que les hommes prennent plus souvent la parole
• nous voudrions aussi surveiller le temps de parole des orgas, pour ne pas trop parler, et pour ne pas avoir non plus la problématique d’hommes qui parlent plus au sein de l’organisation.
• nous voulions limiter le temps de parole à 2 minutes 30 pour que tout le monde puisse s’exprimer, or les compteurs de cette application sont uniquement cumulatifs (on ne voit pas le temps de parole de l’intervention en cours).

J’ai donc commencé par essayer de reprendre le code source existant et de lui ajouter les fonctionnalités manquantes, mais la façon dont le temps est compté rendait cela impossible. J’ai donc finalement développé ma propre application, que vous pouvez trouver à https://www.mentalktoomuch.info/. Elle a les caractéristiques suivantes :

• Mesure du temps de parole par catégorie de genre (ce que faisait déjà l’autre)
• Affichage de la durée de l’intervention en cours et de la limite de temps de parole (par défaut 2:30 mais c’est réglable)
• Catégorisation par statut d’organisateurice ou non
• Affichage des statistiques incluant les prises de parole et le nombre d’intervenants
• Export des données brutes pour faire votre propre tambouille statistique dans un tableur
• Fonctionnement hors-ligne : vous pouvez l’utiliser dans la cave d’un bar sans réseau si vous avez affiché la page avant, ou si vous l’avez « ajoutée à l’écran d’accueil » sur votre mobile (c’est une appli web progressive).

Nous mesurons les temps de parole en café poly depuis environ un an et demi avec cette appli, et nous constatons effectivement que les hommes parlent en moyenne plus longtemps et prennent plus souvent la parole, alors qu’ils ne sont pas plus nombreux en présence. Depuis septembre 2019, nous publions les statistiques sur la page de l’évènement, dans un souci de transparence, et pour faire prendre conscience du phénomène.

Le logiciel est évidemment libre (AGPLv3), vous pouvez donc jeter un œil au code, l’adapter à vos besoins, vous en inspirer pour construire autre chose, etc. N’hésitez pas à me signaler les anomalies que vous pourriez trouver.

L’appli n’a rien de spécifique aux cafés poly, elle est donc utilisable ailleurs. Si vous décidez de l’utiliser dans un autre contexte, ça m’intéresse toujours de le savoir, pour penser à votre cas d’usage quand j’ajouterai des fonctionnalités (si je le fais un jour 😉 ). Donc n’hésitez pas à m’écrire, même si ce n’est en rien une obligation bien sûr. J’espère que ça vous sera utile ! 🙂

Lien vers l’appli : https://www.mentalktoomuch.info/

My experience of Flock 2017

Flock, the annual Fedora contributor’s conference, is now over. It took place in Cape Cod this year (near Boston, MA), and it was great once again.

It started with a keynote by our project leader Matt, who insisted on Fedora’s place in the diffusion of innovation. We are targeting the inovators and the early adopters, right until the « chasm » (or « tipping point ») before the early majority adoption. This means 2 things:

• on one side, we must not be so bleeding edge that we would only reach the innovators
• on the other side, we must keep innovating constantly, otherwise we’re not relevant to our targeted people anymore.

As a consequence, we must not be afraid to break things sometimes, if that’s serving the purpose of innovation.

A lot of the talks and workshops were about two aspects of the distribution that are under heavy development right now:

• modularity: the possibility of having different layers of the distribution moving at different speed, for exemple an almost static base system with a frequently updated web stack on top of it.
• continuous integration: the possibility of automatically running distro-wide tests as soon as a change is introduced in a package, to detect breakage early rather than in an alpha or beta phase.

Seeing where the distribution is going is always interesting, not only in itself but also because it reveals where my contributions would be most useful.

As always, Flock is an opportunity for me to meet and talk to the people I work with all year long, to share opinions and have hallway talks on where our different projects are going (I had very interesting discussions with Jeremy Cline about fedmsg, for example), and to learn the new tools that all the cool kids are using and may make my workflow easier and more productive.

It’s also a great opportunity to help friends on things I can do, and to share knowledge. This year was the first one when I didn’t give a talk about HyperKitty, I guess that means it’s now mainstream 🙂

Instead, I gave a workshop on Fedora Hubs, our collaboration center for the Fedora community. If you don’t know what Fedora Hubs is, I suggest you check out Mizmo’s blogpost and Hubs’ project page. The purpose of the workshop was to teach attendees how to write a basic but useful widget in Fedora Hubs. I wrote all the workshop as an online tutorial, for multiple reasons:

• People can go through it at their own pace
• My time is freed up to walk between the trainees, answer their questions and help them directly
• Attendees can go back to it after Flock if they need to or if they haven’t completed it in time
• It can be re-used outside of Flock (for exemple, by you right now 😉 )

I believe it’s a better way to teach people (see Khan Academy founder’s talk on TED): the teacher’s time is better used answering questions and having direct interactions with attendees, rather than at doing non-interactive things like talking.

There were about 10 people in the workshop, and 4 of them completed the tutorial in time, which is pretty good considering the conditions (other talks and workshops going on at the same time, bandwidth problems, etc.)

Also, I’m getting more and more interested in the teaching / mentoring aspect of software engineering. I like to do it, and I get good feedback when I do. That’s clearly a path to explore for me, although it’s still a bit stressful (but that’s usually a good sign, it means I’m taking it seriously). I don’t want to switch to that entirely, but having some more on my workplate would be nice, I think. The Outreachy program is very appealing to me, it would align perfectly with my other social commitments. I remember there’s also an NGO that offers software training for refugees in Paris, I’ll investigate that too.

 

The workshop on Fedora Hubs at Flock 2017 will be awesome

TL;DR: come to the Hubs workshop at Flock! 🙂

This is a shameless plug, I admit.

In a couple weeks, a fair number of people from the Fedora community will gather near Boston for the annual Flock conference. We’ll be able to update each other and work together face-to-face, which does not happen so often in Free Software.

For some months I’ve been working on the Fedora Hubs project, a web interface to make communication and collaboration easier for Fedora contributors. I really has the potential to change the game for a lot of us who still find some internal processes a bit tedious, and to greatly help new contributors.

The Fedora Hubs page is a personalized user or group page composed of many widgets which can individually inform you, remind you or help you tackle any part of you contributor’s life in the Fedora project. And it updates in realtime.

I’ll be giving a workshop on Wednesday 30th at 2:00PM to introduce developers to Hubs widgets. In half an hour, I’ll show you how to make a basic widget that will be already directly useful to you if you’re a packager. Then you’ll be able to join us in the following hackfest and contribute to Hubs. Maybe you have a great idea of a widget that would simplify your workflow. If so, that will be the perfect time to design and/or write it.

You need to know Python, and be familiar with basic web infrastructure technologies: HTML and CSS, requests and responses, etc. No Javascript knowledge needed at that point, but if you want to make a complex widget you’ll probably need to know how to write some JS (jQuery or React). The Hubs team will be around to help and guide you.

The script of the workshop is here: https://docs.pagure.org/fedora-hubs-widget-workshop/. Feel free to test it out and tell me if something goes wrong in your environment. You can also play with our devel Hubs instance, that will probably give you some ideas for the hackfest.

Remember folks: Hubs is a great tool, it will (hopefully) be central to contributors’ worflows throughout the Fedora project, and it’s the perfect time to design and write the widgets that will be useful for everyone. I hope to see you there! 🙂

React.js is pretty cool

These days I’ve been working on Fedora Hubs, it’s a Python (Flask) application, with a React.js frontend. I know Python quite well now, but it’s the first time I dabble in the React.js framework. I must say I’m pretty impressed. It solves a lot of the issues I’ve had with dynamic web development these last years. And it manages to make writing Javascript almost enjoyable, which is not a small feat! 😉

I’m still wrestling with Webpack and ES6, but I’ll get there eventually. React is really a great way to build UIs. Plus some people are writing the Bootstrap components in React, so this is very promising.