Development Methodologies

December 21, 2010 by gabe · Leave a Comment
Filed under: Implementation 

As a software developer there’s a lot of change all the time. And it’s not always technological in nature. While there are always some changes to platforms, a lot of time is taken up with project management, meetings, planning, documentation and general software development process. I often find myself ranting to co-workers about all the effort spent in chasing the latest fad in process methodologies, rather than ‘just getting stuff done’.
I wonder whether all the time and effort spent on reading about, and attending conferences on the latest nuance of the scrum process, or even spending lots of money and time on tools like Rational Rose is justified. I have run some small software development teams, but can by no means claim any expertise in management, or even proven leadership. But I found simple common sense, determined leadership, and some motivated developers beat systematic, dogmatic process-following every time for any given team. Here’s what I think is important:

  • Motivated, responsible Senior Developers/Tech Leads – financial motivation, willingness to work overtime.
  • Absolute minimum of scheduled meetings – daily status reports by email.  Short bi-weekly meetings with senior managers.
  • Sitting in an open office, to encourage ad-hoc communication, and occasional pair programming.
  • Team-wide design reviews for major features, and ‘one-pager’ design documents for minor features.
  • A manager should have more than 20 reports and work on more than one project.
  • Strictly enforce code style guidelines, and code review process.
  • Emphasize the minimization of lines of code vs. super-clean object-oriented design.  Speed wins.  Squeaky-clean, object-oriented, aesthetically pleasing code is usually outdated by the time it’s actually done.  (Don’t get me wrong – some intelligent up-front design is vital, but should be kept ‘up-front’.)
  • Transparent performance management.
  • Clear quarterly goals.  If goals change, make sure they’re documented properly and simply adapt – formal software development process not needed.

While all the software development processes certainly have a lot of good ideas, I would argue that strong leadership, and financial motivation are an order of magnitude more influential on the success of a project and should be the focus rather than following an arbitrary process.

Sudoku Solver

November 23, 2010 by gabe · Leave a Comment
Filed under: Code, Implementation 

Here’s a little piece of code that I worked on today.  A simple Constraint Satisfaction based  Sudoku solver.  Works for most real scenarios, but will loop forever for invalid configurations. This could be a great little interview question.

function cell(groups) {
    this.value = 0;
    this.groups = groups;
    for (var i = 0; i < groups.length; i++) {
        groups[i].cells.push(this);
    }
}

cell.prototype.isAvailable = function(value) {
    var groups = this.groups;
    for (var i = 0; i < groups.length; i++) {
        if (!groups[i].isAvailable(value)) {
            return false;
        }
    }
    return true;
}

function cellGroup() {
    this.cells = [];
}

cellGroup.prototype.isAvailable = function(value) {
    var cells = this.cells;
    for (var i = 0; i < cells.length; i++) {
        if (cells[i].value == value) {
            return false;
        }
    }
    return true;
}

function model() {
    this.rows = [];
    this.columns = [];
    this.blocks = [];
    this.cells = []

    for (var i = 0; i < 9; i++) {
        this.rows.push(new cellGroup());
        this.columns.push(new cellGroup());
        this.blocks.push(new cellGroup());
    }
    for (var i = 0; i < 9; i++) {
        this.cells.push([]);
        for (var j = 0; j < 9; j++) {
            this.cells[i].push(new cell([this.rows[i], this.columns[j],
                this.blocks[Math.floor(j / 3)  + (Math.floor(i / 3) * 3)]]))
        }
    }
}

model.prototype.init = function(init_cells) {
    var cells = this.cells;
    for (var i = 0; i < 9; i++) {
        var init_row = init_cells[i];
        var row = cells[i];
        for (var j = 0; j < 9; j++) {
            row[j].value = init_row[j];
        }
    }
}

function solve(model) {
    var cells = model.cells;
    // go through each cell and try every value
    var count = 0;
    for (var i = 0; i < 9; i++) {
        var row = cells[i];
        for (var j = 0; j < 9; j++) {
            var cell = row[j];
            if (cell.value == 0) {
                var hasAvailable = false;
                for (var a = 1; a <= 9; a++) {
                    if (cell.isAvailable(a)) {
                        hasAvailable = true;
                        cell.value = a;
                        count++;
                        if (solve(model)) {
                            return true;
                        }
                        count--;
                        cell.value = 0;
                    }
                }
                if (!hasAvailable) {
                    return false;
                }
            } else {
                count++;
            }
        }
    }
    window.console.log(count);
    return count == 81;
}

var sudoku = new model();
sudoku.init([
    [6, 0, 0, 1, 0, 8, 2, 0, 3 ],
    [0, 0, 0, 0, 4, 0, 0, 9, 0 ],
    [8, 0, 3, 0, 0, 5, 4, 0, 0 ],
    [5, 0, 4, 6, 0, 7, 0, 0, 9 ],
    [0, 3, 0, 0, 0, 0, 0, 0, 0 ],
    [7, 0, 0, 8, 0, 3, 1, 0, 2 ],
    [0, 0, 1, 7, 0, 0, 9, 0, 6 ],
    [0, 8, 0, 0, 3, 0, 0, 2, 0 ],
    [3, 0, 2, 9, 0, 4, 0, 0, 5 ]
]);

solve(sudoku);

Dependency Injection

November 17, 2010 by gabe · Leave a Comment
Filed under: Design, Implementation 

Test Driven Development has been around for a while – at it’s code is the use of the Dependency Injection design pattern.  Languages such as Java and C# have many tools and frameworks that make TDD easier and more robust.  Guice comes to mind but of course there are others.  For C++ without the native meta-programming facilities it take a bit more discipline to write proper test-driven code.  However with a bit of practice you’ll be writing cleaner, more bug-free code with no extra effort.

  • Prefer composition over inheritance.
  • All dependencies should be injected using the constructor.
  • Header files should only define abstract interfaces and factories.  This reduces header dependencies, and allows writing mock objects for writing white-box tests.
  • Implementation files implement the interface using inline.

Pretty simple rules!  Since I started using these, I’ve been getting stuff done fast and code quality has improved.  This has had by far the highest impact on achieving results (more so than agile development practices, or arbitrary testing and coding guidelines.)

Here’s a simple example.  Lots of details are missing, but there should be just enough structure to purvey the right idea:

storage.h

class Storage {
 public:
  virtual string WriteRecord(const Record& record) = 0;
  virtual void ReadRecord(const string& name, Record* record) = 0;
};

frontend.h

class Frontend {
 public:
  virtual void HandleRequest(const Request& request, IOBuffer* buffer) = 0;
  static Frontend* New(Storage* storage);
};

frontend.cc

class FrontentImpl : public Frontend {
public:
  FrontendImpl(Storage* storage) : storage_(storage) {};

  virtual void HandleRequest(const Request& request, IOBuffer* buffer) {
    if (request.method == "POST") {
      Record record;
   	  record.name = request.form['name'];
      record.phone = request.form['phone'];
      storage_->WriteRecord(record);
    } else if (request.method == "GET") {
      Record record;
      storage_->ReadRecord(request.query['name'], &record);
      record.Render(buffer);
    }
    request.Reply();
  }

 private:
  Storage* storage_;
};

Frontend* Frontend::New(Storate* storage) {
  return (Frontend*) new FrontendImpl(storage);
}

frontend_test.cc

class MockStorage : public Storage {
 public:
  virtual string WriteRecord(const Record& record) {
    ASSERT(record.name = 'bob');
    ASSERT(record.phone = '12345678');
  }
  virtual void ReadRecord(const string& name, Record* record) {
    ...
  }
};

void TestFrontend() {
  MockStorage storage;
  Frontend* frontend = Frontend::New(&storage);
  Request request;
  Buffer buffer;
  request.method = "POST";
  request.form['name'] = 'bob';
  request.form['phone'] = '12356789';
  frontend->HandleRequest(request, &buffer);
  ASSERT(buffer.find('200'));
  ...
}

Certainly not rocket-science, but if you’re tired of always following the latest fad in software development, this will always bring back sanity to your code.

C++ Collections

November 6, 2010 by gabe · Leave a Comment
Filed under: Code, Implementation 

One of my pet-peeves of the C++ programming world is the lack of a standard collection construct that makes sense.  Arrays, lists, and maps are some of the most-used and most important structures used in programming, and all modern languages actually have given a lot of thought to such matters.  In C++ we’re stuck with STL.  Now, STL certainly works, and it’s widely adopted, but the way in which it only works well with value types, and not reference types it a major pain.  One of the main advantages that C++ has is that the programmer can make decisions on how and when to allocate memory, thus making programs light-weight and self-contained.  When using STL though, adding values to a collection results in a deep copy, putting extra burden on the memory management, and the developer for making sure that all their types are copy-able.  Constructing a simple array of a type that does not have a copy constructor defined does not work.  STL insists on using the copying to initialize it’s elements based on a template.  If there was a way to create new elements in the collection without copying some initial value that would make STL about 80% more useable.  Of course attention would have to be paid for simple type elements that are not initialized by default, some that could probably be handled by a decent C++ programmer.  If not better use Java or C#.

Privacy

October 31, 2010 by gabe · 1 Comment
Filed under: Privacy 

With Facebook and Google being in the spotlight about various privacy matters – intentional and otherwise most people have probably given some thought to where they stand on these issues.  There seem to be two extremes – make people op-in to everything whether that’s taking a picture of your street, or publishing you list of friends, or on the other side of the argument – if you have nothing to hide why worry (or move if you don’t want streetview to take a picture of your house, or change your name, e.t.c.)  Most people fall somewhere in between I would imagine.Both Facebook and Google like to ressure people that they are doing their best and the people who are worried are basically paranoid.  Both have some very valuable, and highly private data in their servers, and I thought I’d try to list some scenarios of what could happen. for either of these companies (I like to pick on Google and Facebook but there are certainly other companies out there with sensitive data – Yahoo, Visa, the Government, Employers, Hospitals, e.t.c. which are not currently under scrutiny about their practices but probably should be.)

The thing that scares me most about Facebook is that it is such a strong identification of yourself.  There may be a time where FB or some other social network becomes your primary identification for anything from payment to travel, to employment purposes.  Anonymity and to a certain extent privacy on the internet will be a thing of the past.

Google has a lot of sensitive information on people mostly from the search engine but also the ads networks, youtube, Gmail.  I’m not worries that this data will ever be exposed intentionally (except maybe to the Government.)  Some of this data is likely being processed and exposed in aggregate through tools such as search trends, search itself, and ad-related targeting algorithms.  It is thinkable that given a good understanding of statistics and number theory a hacker could potentially exploit unintentional weaknesses in the processed data.  Then there’s streetview, and the geo product in general that some find intrusive.  I don’t find it intrusive in it’s current form, but as people can publish their own pictures, and videos for anyone to see — now that is scary.  Google, Bing, Yahoo maps provide a very powerful platform for others to discover pictures.In the end, the data that Google has on people is not as structured and authoritative as what Facebook has.  If Facebook continues on it’s current trajectory, it will easily catch-up with Google in terms of volume and breadth of data, and will become the ultimate ‘1984 platform’.

Data Serialization Tools

October 31, 2010 by gabe · 2 Comments
Filed under: Implementation 

Virtually everyone who codes for a living gets into contact with serializing and deserializing data for transmission or storage.  This affects virtually all platforms whether you’re writing device drivers, or web sites.  The serialization could be a simple encoding for some structured data into a cookie or a url, or it could be full-fledged, optimized serialization for tcp transmission or storage on disk.  Here is a list of the available tools and frameworks that I know about with their pros and cons:

Code-Generation tools

These tools generate code based on an input specification of a datastructure that is to be serialized.  These tools vary in their efficiency and may have cross-platform implementations.  These are generally the most efficient way to go about serializing data as all the repetitive boiler-plate code is taken care of and plenty of tools are client libraries for different platforms are available.

  • Thrift – Apache project, open-source.  Generates optimised serialization code in multiple languages.  Used by Facebook among others
  • Protobuffer – The Google version of Thrift.  Lean-and-mean.
  • COM IDL – Old-school Microsoft COM.  Enough said.  Not providing a link here.

Language-specific Frameworks

Serialization is basically built-in to these languages, making it easy, clean and intutive to transmit and store data on disk.  Platform compatibility, and performance is a secodary goal for most of these mechanisms though.

  • .net WCF
  • Java serialization
  • Interpreted languages: Pytho, Javascript.

Standards

Standards are one way to serialize data by.  Rather than a tool or a language this is a set of rules and specifications to ensure cross-platform compatibility.  Plenty of libraries are available to make using these standards quick and easy.  Standards are highly encouraged when building servers for a large variety of different clients.  However, for single applications or software systems enforcing standards is usually not worth the time or money.

  • XML - The standard of all standards.  Has been criticized about being too wordy in encoding information.
  • JSON – used all over the web.