By: Con Zymaris conz@cyber.com.au
Created: 1999-06-21
Modified: 1999-07-02

Introduction

The following is a presentation first given at the Inaugral Australian Open Source Symposium  oranised by AUUG Inc, ISOC-AU, Linux Users Victoria and SAGE-AU

The Diary of an Open Source Project

Before Open Source

• What did Cybersource do prior to OSS?
• Why did we decide to change?
• Cybersource has always been into Unix/Open Systems
• Most of the developers did Windows apps over the past 10 years. Times were a-changing
• Namely the profit motive went AWOL with the Windows/closed source software
• Windows became a joke to develop robust software for
• We wanted to contribute something back to the OSS community

Some Myths

• No-one makes money from Open Source!
• Not true. In reality few companies make money from closed source software
• More and more money is made by fewer and fewer large organisations. You probably know which ones :-)

• We lose control of our software with Open Source!
• Not true. If you manage the project well and are attentive to the user community, they will assist you in evolving the application, not take it away from you. You are the natural lead

Basic Tenets for Success

• You must need the software
• Otherwise project will stagnate

• Release Early, Release Often
• Listen to your users. They will help drive you forward
• You can only get users involved with useful code

• Release projects under a licence which will ensure greater chance of success
• People are less likely to contribute if they perceive you will gain unnecessarily

Agony of Failure?

• Many Open Source projects do not succeed
• Don't sweat it. Make your code available to one of the web free-code repositories like http://www.mibsoftware.com/reuse/

• Natural Selection
• What makes Open Source a solid software engineering process is that some (many?) projects & code modules are culled. Think Darwinian selection and fitness-landscape applied to software :-)

Increasing Chances of Success

• Use the Unix Philosophy
• Small apps which do one thing well
• Which are linked by the standard Unix STDIN/STDOUT piping. The only OLE which works!
• This allows for small discrete code units to be easily written and debugged well. Contrast huge Windows-based bloatware and DLL-hell

• Base your project on as much existing code and components as possible
• For example, use existing functionality like the GNU 'locate' rather than recode your own

• Use simpler, multi-platform scripting systems
• Languages like Perl, Python, PHP, Tcl/Tk allow you to deploy your app on many OS'es
• They are simpler for many people to code with than C++ which allows greater scope with getting your project off the ground fast
• Greatly reduce the development cycle through having no 'compile' phase
• Allow you to string together existing Unix commands to achieve your codes aims, easily and robustly

Web Deployed Apps

• Why we chose a web-deployed App for our project
• Think of it as a completely allergic reaction to the problems associated with Windows-based fat client applications. DLL & OCX problems. Registry problems. Install and file maintenance problems
• We were finding that more and more of our clients were thinking seriously about web-based apps. Our people needed skilling in these technologies and tools
• We believe that most line-of-business apps will be web-based in the coming 3-5 years

Our Project -- CybOrg

CybOrg - Cybersource Office Organiser
 

• What is it?
• Web-based back-office information system
• Contacts
• Reminders
• Call Centre
• Document Manager/Web File Manager
• The ability for it to be given to end-users so they can create their own web-based data-centric apps. Almost like MS-Access in a web-browser

Technology Used - PHP

http://www.php.net/

• Why?
• Simplest of the serious database capable web-backend scripting systems
• Has DB support for many SQL databases, including the ones we were interested in
• speed! It's fast to code in and fast to run
• designed for web work, unlike Perl & Python
• It's open source!
• Also, mod_php works like a charm as an Apache extension

MySQL & PostgreSQL

http://www.mysql.com/
 

• Why?
• It's the fastest web-style (i.e mostly read-only) DB around
• Benchmarks show it to be around 14 times faster than Oracle for queries
• It's tiny to deploy; wonderfully easy to maintain
• Has some missing functionality, but little which impacts this type of application
• It's mostly open source

http://www.postgresql.org/
 

• use this for heavier 'transaction-oriented' code
• It's fully open source

Apache

• The web server of choice. Why?
• fast enough. use in combination with thttpd if speed is vital
• Multi-platform
• open source
• has excellent extensibility through the mod_ interfaces
• has a cool name

Linux development platform

• While we are working with tools that could just as easily have been installed on FreeBSD or Solaris, we chose Linux because:
• It's got the right community for this type of project
• the developers involved (unlike other segments of our company) know it better than anything else
• that's what was installed! Don't waste time rebuilding a development platform

ht://Dig

http://www.htdig.org
 

• A great full-text search engine. Think Lycos for Intranets
• Keep with the philosophy of using what is out there, and is compatible with your choice of development methodology and licence requirements

CVS - Revision Control

• Why CVS?
• It's open source. This is important for long term availability of applications.
• In the past, we've been burnt by using proprietary, closed source versioning tools
• It's on all the platforms we are interested in developing on
• It allows for more complex development processes such as multi-programmer merging
• More Information?
• http://www.cyclic.com/
• http://www.loria.fr/~molli/cvs-index.html

The IDE? A Text Editor

• As many of the developers on the project came from a Windows background, where IDEs and Visual development tools are the norm, what did we use for development of CybOrg?
• VIM for power users
• pico for the simpler minds (Like yours truly :)

• Why?
• simplicity and power
• availability of editing interface from a 'distance'
• this was important as development often happened remotely, after hours, via SSH

First Steps

• Planning
• What exactly are we making here?
• Keep the initial plans brief, else no-one will read them
• Keep meetings brief, and preferably via email
• Allocation of tasks
• Someone needs to 'project-manage'; not everyone can code :)
• Documentation is important.
• Get agreement from all project members on use of CVS, simple coding standards and directory tree structure

Licence Choice

• GPL? BSD? Artistic? MPL?
• If you're planning on making commercial 'Pro' versions, go with BSD or MPL, else stick with the GPL
• What do the core project members prefer?
• Each licence model has its 'nuances'. Read them, and any commentaries carefully. Don't go off and create (yet another) new licence!
• Don't get religious about it. It's important, but don't let it become a sticking point for the project

Documentation
 

• As with all else, KISS! Text and HTML the best way to go. Avoid proprietary 'Word'® processors!
• Why? People on different platforms cannot read your docs, or may be dissuaded from doing so by your choice of doc-format
• There are a wide variety of more powerful (and complex) documentation prep & deploy systems out there if you need them
• Check out the 'Open Source Writers Group'
• http://www.linuxchix.org/techwriters/

Mailing Lists

• A Mailing list is a must for developers of OSS pojects.
• They are also very important if you hope to foster a 'user community' who will assist in ongoing debugging and idea generation
• Mailing list manager software?
• Majordomo, Mailman etc
• For those who can't establish their own list(s), there's Topica, which provide a nice, easy, archived, free mailing list service
• www.topica.com

CybOrg Timeline
 

• Now for some details about our project
• Project started in February of 1999
• 5 main people involved initially. The task breakdown was as follows
• 2 core developers (Darryl Ross & Andrew Delboy)
• 1 install script writer (Alvin Sim)
• 1 tester (Marco Grigull)
• Documentation (Con Zymaris)
• Screen mockups and data-diagrams happened first, followed by building the devel environment
• Research was done to determine if a project like this had been done beforehand: namely a PHP/Web based general purpose database independant business info system
• We found that some components had been done, we checked their licences, and if applicable, we incorporated them into CybOrg with attributions. Changes we sometimes needed to make the DB backend generalised. We wanted to avoid 'needs MySQL' or 'needs Oracle'
• We produced the first working module, the web-based file/doc manager
• We made a decision to use existing tools extensively. Examples are:
• locate: used for file searches
• ht://dig: for full-text searches
• We decided to make CybOrg as modular as possible. Why?
• So that each module could be focussed on and written as a way to maintain momentum
• because it makes it easier to build a customised intranet back-office suite for users. Just plug in/out the modules you want
• So that others could write modules easily
• We created a home-page for the project
• http://www.cyber.com.au/cyber/product/cyborg
• We established our mailing lists (on Topica) for:
• developers: cyborg-devel@topica.com
• users: cyborg@topica.com
• We incorporated Benjamin & Jim Suto's E*Reminder project as our Reminders module
• We were part way through the web-based MS-Access clone

Losing Momentum

Round about now we had several 'working' modules, and we were part way through the biggest and most complex component, the db_forms (i.e MS-Access clone) code. Work ground to a halt as the main developers were seconded onto other projects for clients, or were stretched with existing 9am-6pm commitments

How the distractions helped

One advantage of this 'commercial' services work was that we were able to use and deploy parts of CybOrg. This had the benefits of saving time and money for the clients involved in getting their projects to fruition faster. Advantages for us were that we could trial the modules we had coded thus far with 'real world' (read 'big company') data. None of the client projects were intended for closed-source release, so our GPL'd code didn't cause concern.

How the distractions hindered

The biggest problem that the distractions have engendered is that while work on the core code continues, communication between the project members has become harder. This has had the following negatives:
 

• Work on the install scripts has ground to a halt. This has meant that we have not released the source in an easy to deploy manner, and only a simple tarball will be made
• It's harder to get responses from the developers on matters relating to tech-docs

Were are we now?
 

• We've got more confidence in our code and project development methodology
• We need to work out a method to avoid getting 'bogged-down'
• We're probably a few weeks away from making the first full suite release. This is likely to be 5-6 modules, plus the db_forms engine
• We're positive that new developers will arise to help us out, but we're unsure of how to include them into the decision tree

Surely OSS doesn't need marketing? Wrong!
 

• To increase the likelyhood of your project becoming a sucess and producing a self-sustaining community, you must get the word out there. How?
• Freshmeat: http://freshmeat.net/
• LinuxToday: http://linuxtoday.com/
• Linux Weekly News: http://lwn.net/
• As with everything, the better you get the word out, the more people will be available to spread the word about your open source project!

Why Not?

• will possibly gain many developers and testers, world-wide, free
• you can use GPL'd source code to help boot-strap your project and increase the likelihood of on-time, on-budget
• you will get kudos from the clued-in people, world-wide

Your Turn

While no one is keeping tabs, you probably know that if you use OSS extensively, good Karma dictates that you must contribute somehow. How?

• Start an OSS project; something you have an interest and need in
• Fund a small module; if you really must have some specific functionality that's missing, pay to have it developed
• Write documentation: HOWTOs, FAQs, web-page conglomerations on other OSS apps, etc. Good Luck!