The Open Source U

As some readers will know, I’ve been working to study the architecture of FireFox. One thing I realize is that I’m not as familiar as I should be with some of the common open source tools, or what I consider to be the common open source tools.

Below is a list of tools that are worth learning (in my opinion). I’ve also included a short description of why it might be worth learning them. (The list is in no particular order)

If you have an interest in open source, it will not hurt to get a quick base understanding of each of these. I’m not saying you (nor am I trying to) become an expert in each of these tools. However, having a basic understanding of syntax and function should save time and headache while trying to understand a project.

1. Linux: The concepts in and around Linux are often used on other open source products. I think Open Source developers tend to stick with using open source software. So, there is a link there.
2. Bash: The de-facto standard shell for Linux (as far as I can tell). Knowing the basic usage of bash can save you time and confusion.  Certain scripts can depend on feature of your terminal interface. The Mac OS X ships with a version of bash, which is good to know…
3. GCC: This is the GNU Compiler Collection and is often a requirement to build open source packages.
4. Make: This is a part of the GCC but I want to make special mention of this because knowing how to read the script files and error messages can help diagnose an error.
5. C / C++: Low Level libraries are often written in C or C++, even for an otherwise Java or Python based program.
6. Python: Python is sometimes used in conjunction with Make to check for build dependencies, verify (build) requirements, or many other possible things.
7. Perl: Often used like Python, from what I understand but I’ve yet to learn much about it.
8. Subversion: This is the most common open source VCS software in use (based on my own observations)
9. Mercurial: One of the two popular DVCS systems. I’ve noticed more and more open source projects switching to DVCSs, so a basic working knowledge of Mercurial and Git is helpful.
10. git: The second of the two popular DVCS systems.

If you’d like to contribute to the list, please leave a comment below but please ensure you include a fair reason.

This is a follow up post to a series I’ve been posting. The previous post to this posting was “Firefox – Found time to Try again!”

I finally was able to get Firefox to build on my Vista machine. It has given me a lot of trouble and I’ve had to troubleshoot several times. Relative to my previous post, I was last held up on a “NSInstall” error. NSinstall.exe would report “Bad File Number”.

The solution was reducing the directory depth. I had my mercurial repository in a sub folder of my documents folder. Specifically C:\Users\Frank\Documents\hg-repos\mozilla-fv-expirement\. I moved the directory to C:\mozilla-fv-expirement\ and re-ran the build script. It did it’s thing and I was able to fire up good-’ol firefox.exe from the [...]\dist\bin folder. I even checked my gmail account on it.

Now where?

I felt that before I started hacking and understanding the source code, I should be able to build the program. That way, if I make a significant change I can see how the code base reacts to it. Though, my intentions are not to hack but to understand how it works. The ability to drop print statements in should help with this… maybe…

Additionally, I hope to contribute to the Mozilla Litmus QA system. I might as well — I did take the time to build the trunk.

My attempts to build Firefox on windows continue today. If you recall , previously, I’ve had great success with Mac OS X which I tried by chance but I’ve had problems while attempting the same thing on Windows.

This experience has made me decided that my troubles with building open source program in the past has stemmed from my usage of non-unix based operating systems — Windows — and my lack of understand about make, autoconf, python and perl.

Ok, lets go….

I’ve updated to the latest tip of Mozilla Central and at this very moment, I’m at revision 80f4cdc242b9. I expect to be working with this revision throughout the remainder of this post. Though, in general the exact revision shouldn’t matter if you are attempting the same thing.

I’ve just tried to build with the command make -f client.mk

I’ve received an error and while writing this I just thought of why. It’s stupid reason but one is bound to make mistakes like this. Simply put, I forgot the very required mozconfig file. I cloned my “clean” repository to a new directory to start over and I forgot this file. For records (and search engines) the exact error I received was “configure: error: –enable-application=APP was not specified and is required.”

So, now lets actually follow the directions since we are not experts.

I’ve reread the MDC about building Firefox and I’ve created a .mozconfig files. I’ll put it below for my and your reference. Don’t assume you should use this — check out the Mozilla docs. They are very straight forward.

# My first mozilla config - https://developer.mozilla.org/en/Configuring_Build_Options

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@

ac_add_options --enable-application=browser
mk_add_options MOZ_CO_PROJECT=browser

# Use the follow two for debugging (if desired)
# ac_add_options --enable-debug    
# ac_add_options --enable-tests

The # is the comment indicator and as you can see, I’ve setup but not enabled my options for debugging. I hope to use these later but for now I’ve decided to disable them.

Given the above .mozconfig file, I’ve attempted to rebuild and run in to an apparent known issue with building on Vista. Mozilla has done a beautiful job document the reason and how to fix it.The exact error is: “configure: error: System header oleacc.idl is not available” but right afterward the URL is given.

To correct this problem, Mozilla has listed three options. I’ve elected to attempt the “Use Windows 2008 SDK” option. I wasn’t sure which to pick but I decided that the 2008 SDK seems to be supported as the site states “Unfortunately, the Vista or Windows 2008 SDKs are required to build Mozilla”. I wish the page made a recommendation or gave pros and cons but perhaps it doesn’t matter…

The next error

After getting the Windows 2008 SDK installed and up to date, I received another error. This time the error read “nsinstall: Bad file number”. I’m not sure where to go from here, so I posted at the usenet.

The prevalent solution was to run as administrator, according to these two articles:

• Penelope Developer Page
• kevin brosnan dot net

This hasn’t worked for me; I’m still trying to figure this out. Apparently, my success is not going to come today so I’m setting this aside for the time being.

In the hopes of continuing soon, I’ve posted two questions:

• Usenet
• Stack Overflow

This is a follow up post to FireFox — Getting Started. Consider this more of a status post to show my progress in the previously linked post.

I’ve orientated myself on Mercurial a bit but I still have a lot to learn. I must state now that it is completely different from Subversion (SVN) and CVS. If you are new to systems like Mercurial — otherwise known as Distributed Source Control Management systems, I suggest you study up on them. The Mozilla team has put together a great starting out reference which is here and the Mercurial team has put together a great book here.

You can get information on getting the latest tip of “mozilla-central” from the following web page:

https://developer.mozilla.org/En/Developer_Guide/Source_Code/Mercurial

One tip I do want to leave here for others is that you should clone the mozilla-central repository twice. Once from the http site and once from the new local copy.

The first one, which I call clean, is a exact clone of the mozilla-central from Mozilla. I then clone this local repository and work in this new clone. I call this local clone working. With this setup, if you massively screw up your working directory, you can always re-clone your clean copy. I believe I read this in the Mercurial book.

My First Build

This series of blogs posts is actually suppose to be focused on the construction and (in some ways) development of FireFox. But, for me the first step to understanding a given project is to be able to compile or build it and run it. If you can’t build or run it, you can see what your hacking is doing to the results.

I’ve been able to successfully build the source code, however not on Windows as I had hoped. At least not yet. I’ve successfully built it on my Mac OS X machine.

Anyway, I continue with my quest to build on Windows as I suspect most of my research will be on my Windows Vista machine.

I’m not going to include how to build Firefox here. The Mozilla team has done a wonderful job documenting how to build on each environment and furthermore, they have set everything up to be newbie friendly.

My only advice at this point, is to go from top to bottom — don’t skip steps and install everything that they ask. Quite simply, if you skip around without knowing what you are skipping, your build won’t build and the error messages can’t tell you what you are missing. You end up reviewing the directions from top to bottom again.

Again, here are the Simple FireFox build instructions from Mozilla.

This site was established with the my express desired to learn how  certain major open source software packages work; this was to be a personal endeavor to broaden my horizons.

As it turns out, I ended up enrolling in graduate school. I’m pursuing a Master of Science in Software Engineering — something I’ve been planing since my B.S. (which is as represented).

I lost my focus on this site but I don’t want to. I want to regain that focus and this is my first post to that end.

This post is the information I gather while setting up to review and understand how Mozilla’s FireFox works.

Getting Started with Firefox

A while back, before I had even started writing this post, I posted a question on StackoverFlow.com about how to learn about how FireFox works. I’ve received some great answers.

The answer I chose to accept was from jbinto; who gave a wonderful detailed answer and list of resources to pursue this. I’ll be utilizing these resources along with my own method of stumbling though the code.

I’ve started with the Windows Build Prerequisites page since I’ll be building on windows (vista). I imagine my greatest audience will be Windows users and hence I’ll stick with windows. But I do use Linux and Mac OS X too, so if you have a question about these environments, post a comment and I’ll try to address it.

I’ve downloaded and installed the package. It apparently includes everything you need to build FireFox — Including the Source Control Tool and diffing utilities and such.

I want to work on the latest tip; but apparently, the FireFox team uses Mercurial as their source control tool. I’ve never used this tool so this is good and bad. I can learn to tool but I  have to spend the scarce resource of time to do so. I’ll be taking a short deviation way from my current plan to learn the Mercurial basics.

I won’t be posting much on how to use Mercurial as there is plenty of documentation. Anyway, off to learn about Mercurial — I shall continue with this article series once I’ve obtained the tip of FireFox.

My plans for this site are not quite clear. I feel that I’ve got a very clear objective but the plan on how to achieve that objective is still a bit fuzzy.

As I describe in my about page my mission plan (for a lack of a better term) is:

Understand the design, architecture and implementation of certain major Open Source software for my personal and professional growth.

On this site, I do intend to walk though the dissection of certain open source software. Additionally, I’m going to walk though and explain by example certain design patterns as well as post example software.

Lately, my main focus has been software architecture. That is, how is software put together. This is as opposed to how certain functions work or operate. I’m looking at the high level of software design.

Head First Design Patterns Book Cover

I’m currently reading Head First Design Patterns. I love the book and this book is probably the main reason for my interest in overall software architecture. The architecture is something that I’ve not spent much time on in my career as I mostly put together software segments rather then the overall software.

Recently, I’ve become responsible to design the architecture of a existing project that is in terrible need of being re-done. I hope to learn things from the book and from the internet to help me in this effort.

As for the software dissection side of things, I’m thinking of starting with Firefox. In fact, I’ve actually started as I’ve always wanted to dissect Firefox; it intrigues me. A little bit ago, I posted a question on Stack Overflow titled: How Does FireFox Work? Source Code Walk Through?[^]

Check out the post for some of the responses I got. The one that is marked as the answer is what I’ve decided to follow.

I hope to start writing up my findings and understandings later this week. Though, with work my schedule can get dicey.