Intro to FOSS Project Anatomy (Activity)
Contents |
Preparation:
Description | Introduce participant to structure, processes and development tools used in FOSS projects. |
Source | |
Prerequisite Knowledge | None |
Estimated Time to Completion | 60 minutes |
Learning Objectives | |
Materials/Environment | Access to Internet/Web and web browser. |
Additional Information | Producing Open Source Software] by Karl Fogel |
Rights | ? |
Background:
FOSS projects have a distinct culture and set of tools that support project development. The form of the culture and the specific tools vary somewhat across projects, but there is significant commonality such that many FOSS developers migrate easily among FOSS projects.
Now that you have an understanding of the communication tools used in FOSS environments, this activity will provide a high-level understanding of some of the culture and development tools that you will use when participating in a FOSS project. The goal of this activity is not to provide depth in any one tool or aspect of culture, but to provide a high-level view of what a typical FOSS project looks like.
Directions:
Read the information below which provides a foundation for understanding a FOSS project. In this section we provide basic definitions for the common aspects of FOSS projects. The Guided Tour will walk you through an HFOSS project and highlight some of these aspects described below. We begin with a discussion of community as FOSS development is as much about community as it is about code.
Community - As you have discovered, FOSS projects operate on the basis of a meritocracy where a person defines their position within the community based on the merit of their contributions to a project. These contributions do not necessarily have to be code. As an individual continues to make valuable contributions, that person will be given (or will take on) additional responsibility.
There is a typical progression of participants in a FOSS community. Most FOSS developers start as users of the application. They then identify some small change that they want to make and make the change. If the change is accepted, they progress to making a larger change and eventually become responsible for a portion of the project. A committer is a developer who has risen to the level in the meritocracy where that individual is responsible for committing any changes submitted to the code base. This is a large responsibility as the committer must review all changes and make sure that the change is of sufficient quality to be incorporated into the code base.
Leadership - You may be thinking "But how are decisions made?" In many FOSS projects, there is a "Benevolent Dictator", AKA "BD". This is the person who has the final say on a decision. And while this person has the final say, they typically rely on several or more developers who have risen in the meritocracy to the point where they have the ability to commit significant changes. As such, the benevolent dicator doesn't actually dictate much, but instead comes to a collaborative agreement with the major developers who have expertise in the area in which the decision must be made. Most BDs actually do not like making decisions by fiat and are reluctant to put their foot down.
As projects mature, they frequently migrate from a BD model to a democratic model where decisions are made by consensus. A consensus is a decision about an issue that everyone agrees is a reasonable solution. The consensus is usually reached via a discussion about a particular issue and how to solve it. The important thing about consensus organization is that when consensus is reached, it must be posted publicly.
Forking - One of the imporant cultural and technical aspects of a FOSS project is its forkability. Forkability is the ability for anyone to make a copy of the code and start their own project based on that code. That new project is known as a "fork" of the old project. For instance, LibreOffice was forked from OpenOffice. When a project is forked, the source code is copied to a new project and usually some members of the original development team leave the original project and migrate to the new project. Forking is not an ideal situation as it may result from dispute with in the development team that cannot be resolved.
Communication - FOSS projects use common communication tools. You have already been exposed to wikis and IRC. In addition, most FOSS projects have one or more mailing lists. In addition, bug trackers and version control systems (discussed below) are also frequently used for communication.
Roadmaps - FOSS projects need some way of defining the future of their project. Most FOSS projects have a Roadmap or Blueprint that does this. The Roadmap may be a detailed schedule of releases with the specific bugs fixed and enhancements added for each release detailed. In smaller projects, the Roadmap may simply be a list of the next enhancements to be added to the project.
Releases - A "release" happens when a project has been developed to the point where the developers want to distribute the code and documentation. Releases are typically planned based on the roadmap for a project. Software development is typically "frozen" at a particular point in time so that testing of the software may proceed before the release. This does not mean that development stops. Version control is used to manage different branches of development that allow some developers to continue to work while others test the code to be released.
Releases are numbered and the convention is to use 1.X for major releases. Numbering 1.X.x is used for more minor issues. The next major architectural change would be in 1.2. A complete upgrade of the code (e.g., from python 2 to python 3) would result in an entirely new number, e.g., 2.1.1.
Repositories - Most FOSS projects save their code in a repository (AKA repo). A repository is simply a place where software packages are stored and from where they may be retrieved and installed. Some projects use a web-based common repository such as Launchpad or SourceForge, or a local repo. The repo is where users go to download a software application.
Packaging - FOSS projects are distributed as source code. In fact, access to the source code is one of the major benefits of FOSS. The source code should be distributed in a standard format (e.g., tar compressed by gzip for *nix).
However, code that is distributed in this manner must be downloaded, unzipped, untarred, and then installed using a manual process. As a result, most FOSS projects create "binary" packages to support easier installation. In this case, "binary" doesn't necessarily mean compiled. It means that the package is pre-configured to be installed on a computer without having to build the project from the source code directly.
Upstream/downstream - Note that development in a FOSS project is not entirely linear. There are typically many people working on the project simultaneously. The main branch of the project is said to live "upstream" of all efforts to make changes. The upstream version is the main version. Developers "downstream" check out a copy of the code to work on. When a change is made, that change is pushed upstream to the main branch so that everyone is working with a consistent code base.
Upstreaming and downstreaming may also refer to the build process where packages are built for release. In addition to upstreaming in an individual project, a project may depend on another project. If project A depends on project B, then the release of the upstream project B must be complete before the release of project A can happen.
Version Control - Most FOSS projects have multiple people working on the project simultaneously. Developers may even be working on the same area of a project at the same time. A version control system is a system for controlling the revisions made to software. Version control software tracks and controls changes to a project's files. Most FOSS projects use some form of version control to control change in the software. Commonly used version control systems include git, SVN, CVS, and Maven. The version control system provides both a method of communication as well as a way of coordinating changes in the software. There is a learning activity later on version control. Additional reading: | Wikipedia page on revision control.
Trackers - In most FOSS projects, there is a need to keep track of all of the bugs found as well as the new enhancements suggested. A bug tracker (AKA issue tracker) is used to track change requests for a project. These change requests could be bug fixes, new feature requests, patches from outside developers, and more. A bug tracker allows issues to be logged, described, prioritized, reproduced, diagnosed, fixed and tested. There is a learning activity later on bug trackers. Additional reading: | Karl Fogel's section on Bug Trackers.
Working Material
- Stoney
- Peter
- Darci
- TO THE ACTIVITY DEVELOPER
- A directed walkthrough into a specific project. Participants answer questions engaging with terminology.
- TO-DO LIST
- Pick project: Peter
- Gather links
- Draft walkthrough
- Candidate Projects
- OpenMRS
- Ushahidi
- Sahana
- Mifos
- Sugar (one-laptop-per-child)
- HFOSS Project list: http://xcitegroup.org/softhum/doku.php?id=g:hfoss_and_oss_projects
- https://developer.mozilla.org/en-US/docs/Introduction
- https://developer.mozilla.org/en-US/docs/Developer_Guide
- Topics
- What is a forge?
- Not all open source projects exist in a forge.
- The status of a project may be anywhere in the planning to mature stages.
- Regarding community - What major roles/teams exist within the project?
- releases: find version number system for project.
- code repository policies:
- packaging
- upstream (no artifact)
- downstream (no artifact)
- trackers: find the tracker
- tickets: look at some tickets
- roadmap, etc.: find the roadmap. How are releases managed? How
- Reference materials
- First couple of chapters in "Productively Lost" ? http://teachingopensource.org/index.php/Textbook_Release_0.8
- Mel has a good overview of some parts in "Roadmap Merge" activity here:http://mchua.fedorapeople.org/posse/POSSE-booklet.pdf
- http://en.wikibooks.org/wiki/Open_Source
Peter's Silly Attempt
- Pick a project: Lighttpd. http://www.lighttpd.net/
- Why? Small enough to get a handle on rather quickly (especially since they have a semi-documented plugin system **http://redmine.lighttpd.net/projects/lighttpd/wiki/HowToWriteALighttpdPlugin so you can build a small piece before building a big piece), important enough since used on popular websites.
- Tracker/Tickets: http://redmine.lighttpd.net/projects/lighttpd/issues
- tracks both bugs and features
- ticket status from new to feedback to actual patches
- locates most issues in modules of architecture
- Roadmap: http://redmine.lighttpd.net/projects/lighttpd/roadmap
- outlines which issues are going to be fixed in which release of the project
- stratifies issues into what's going into the next release and what's going to be addressed (a lot) later
- release numbering is 1.4.concrete for next release, 1.4.x for issues that's not too far off and may lead to smaller architectural changes (but might never get fixed if things go badly), 1.5.x for their next architectural vision which will overhaul the entire project
- Upstream/Downstream: ?
- uses svn/bzr?
- general definition: upstream is where THE source for the project lives, what will get released; downstream is where YOU work and try to fix things; patches go from downstream to upstream, not sure how in lighty yet
- Releases: http://blog.lighttpd.net/ and http://redmine.lighttpd.net/projects/lighttpd/wiki/Releases
- announcements are here but where the plan is I don't know
- two tracks: release candidates vs. stable releases; candidates are testing fixed issues among a wider group of people, not recommended for stable deployments
- Code? http://redmine.lighttpd.net/projects/lighttpd/repository
- IRC: irc://irc.freenode.net/lighttpd
- Community: http://redmine.lighttpd.net/projects/lighttpd/boards
- prefers IRC and forums
- has mailing lists as well http://redmine.lighttpd.net/projects/lighttpd/wiki/MailingLists
The Sugar Labs Project (http://sugarlabs.org/)
Read the information found here to get an overview of the goals of the project and the expectations of contributors.
Community -- In the first paragraph, they indicate the the community is organized into teams. Follow this link to the teams page. You will note that there are a wide variety of teams, each with a distinct responsibility. Follow the 'Contacts' link (found in the green option bar) for each of the following teams and summarize the information you find there on your faculty wiki page. For example, are there any commonalities? Is there something distinct for each type of team?
- Activity Team
- Development Team
- Documentation Team
Tracker -- The Sugar Labs bug tracker can be found here. On your wiki page indicate the types/categories of tickets listed on this page.
Repository -- http://git.sugarlabs.org/sugar-base
Release cycle --
- Upstream/Downstream: diagram with explenation
- Communication:
- Roadmap: ???
- Repository strategy: https://wiki.ushahidi.com/display/WIKI/Our+Git+Repository
Darci's Excellent Attempt
- Project: Sahana Eden - http://eden.sahanafoundation.org/wiki
- Tracker/Tickets: http://eden.sahanafoundation.org/report
- Major Players: http://eden.sahanafoundation.org/wiki/WikiStart#WanttoContributetoSahanaEden
- Forge/Repository: https://github.com/flavour/eden/fork_select
- Release cycle / version numbers:
- Communication: #sahana-eden Main Sahana Eden Channel;