Posted on

Continuous Integration

Continuous Integration

What?

Continuous Integration (CI) is a process that allows developers to integrate, in a repository share by the entire team, code developed by himself, at least once a day. At an Utopian Universe, those integrations should happen several times each day, basically every time some code is “pushed” into the repository. This action will trigger the automated process.

Why?

Agile methodologies demands code to be always “ready” to be delivered. This process of continuous integration of the entire team’s code, in the product/project code base, allows an early error detection.

In a Waterfall process, each code delivery might took several months to happen. As you can imagine, in that time, it was usual to set a “War Room”, where the entire development team would be, in the company of the entire operations team, QA team, product team, and management team. The delivery process were a terrible chaos, a real nightmare, and it could take hours to deliver (don’t bother asking about any issue that might occur, the responsible team had to fix it “pronto”).

In fact, back in those days, imagining that each code push might trigger a battery of tests (unit, integration, acceptance, …) and have immediate feedback about the possibility of the new code breaks any existing functionality, it was unthinkable. A true adventure !!

How?

There are several available options for those that wish to start using a development pipeline with continuous integration. Probably the most notorious is Jenkins. An open-source tool written in JAVA. To those that do not want to add another tool to their setlist of tools to administer/manage, there are some SaaS solutions. Nowadays, at log, we use the CloudBees’ SaaS Codeship.

Once configured, every time a new push to a Git repository happens, it will trigger an automated process where a virtual environment, with characteristics specified by ourselves, is set and where the build process will happen, followed by the entire tests battery – unit, integration, acceptance, mutation, …

In any scenario, if a problem occurs, the entire continuous integration process fails and we will have a complete report. It is important to state that when the process fails, there are no possibilities for a deployment – automatic or manual – to whichever environment.

There will be no eXtreme Programming without this important process !!

Posted on

WordCamp London 2018

WordCamp is a conference that focuses on everything WordPress.
WordCamps are informal, community-organized events that are put together by WordPress users like you. Everyone from casual users to core developers participate, share ideas, and get to know each other.

As you may guess by the statement transcript from the “WordCamp Central“, WordCamps are moments for sharing, not only share knowledge and experiences, but also an opportunity to extend ones’ network and form alliances.

log, the software agency to whom I work for, will sponsor the WordCamp London and will be present to showcase our nearshore development services. I, with great enthusiasm, will take part on the entourage. It will be a real pleasure to spend a couple days talking with fellow WordPress developers and, with great pried, promoting the quality of the code developed by my team.

I still believe in the possibility of promoting and evangelizing the usage of the best practices suggested by eXtreme Programming. For sure this will be another great opportunity to debate the subject.

Will we meet there?

Posted on

Pair Programming

Pair Programming

The pair programming process is a work methodology advocated by eXtreme Programming (XP). As the name implies, each programming unit consists of two developers on a single computer. We may incur in the wrong thought that it is a waste of human capacity, since there are two elements to accomplish one’s tasks. However this thought could not be further from reality.

It’s expected that a paired team will produce the same amount of code as would be expected from two individual developers, however with the added value of the code having higher quality. In an earlier article I wrote about the importance and the role of code review in software development. A process that also consumes time in the present moment but that allows us to gain time in the future through the greater maturity and quality of the code that will follow for production environments. If one chooses this methodology, it can be assumed that the code revision will already be present in the code developed by the pair of developers.

Each developer will have a specific role – the “driver” and the “navigator”.

This means, one writes code while the other reads and anticipates issues. The best matching occurs when the keyboard goes hand in hand without any of the elements clinging to their pre-set role. As you may see, pair programming has much of soft-skills has of technical skills. It is not easy for a developer to open hand from his opinion before testing it. Like all soft-skills, it is possible to develop it but it requires time and openness from the individual.

“Since I’m going to have two developers together, it’s best to join a senior together with a junior, so he can learn”.

You must be really careful with this line of thought. A mentorship relation is very different from a peer relation. If productivity is not meant to be reduced, the team should be made of competent, autonomous developers and at the same level.

There are companies that prefer to define the roles as “developer” and “tester”, this means that the “developer” should write code to pass the unit tests written by the “tester”. The team will change roles often and sometimes even in the same day. As you may figure out, this would be impossible if both developers weren’t at the same level.

Another indication that eXtreme Programming gives, “move people around”. It is important that pair programming teams switch elements regularly. With this we will prevent the emergence of “islands of knowledge”, which in the future may lead us to a technical debt if the holders of knowledge decide to leave the company – a phenomenon feared by any technology manager.

Thus we may realize that pair programming paired with an internal developers rotation we will obtain greater:

  • continuous flow of development and delivery,
  • dissemination of knowledge of the project’s codebase,
  • quality in the production code,
  • decrease of possible technical debts.

 

Posted on

Code Review

Code Review

IBM championed the code review process since a long time, in fact since 1974. It was initially defined as a “code inspection”, a formal static analysis process to evaluate if the software was full-filling the requirements. This process was slow but, even so, IBM enabled to obtain significant quality improvements in the delivered code.

The company (IBM) nearly doubled the number of lines of code shipped for System/370 software products since 1976, while the number of defects per thousand lines of code was reduced by two-thirds – Michael Fagan

Thanks to collaborative version control tools, it was possible to speed up this process, making it a daily routine process. In my team, in every single day, each software engineer reads, reviews, evaluates and comments on a colleague’s code. The ultimate goal is increase trust in the shippable code, reducing the number of possible bugs. The review process implies that the code may be updated and reviewed again. This exchange – between the author and the reviewer – should happen the necessary number of times until the demand level of quality is secured.

There aren’t specific rules on how this process should work. With the dissemination of this process, the good sense prevailed and some good practices emerged from different sources.

SmartBear Software company published a small white-paper with 11 good practices for an effective code review process:

  1. Review fewer than 200-400 lines of code (LOC) at a time.
    • More then 400 LOC will demand more time, and will demoralise the reviewer who will know before hand that this task will take him an enormous amount of time.
  2. Aim for an inspection rate of less than 300-500 LOC/hour.
    • It is preferable to review less LOC but to look for situations such as bugs, possible security holes, possible optimisation failures and even possible design or architecture flaws.
  3. Take enough time for a proper, slow review, but not more than 60-90 minutes.
    • As it is a task that requires attention to detail, the ability to concentrate will drastically decrease the longer it takes the task to complete. From personal experience, after 60 minutes of effective code review, or you take a break (go for a coffee, get up from the chair and do some stretching, read an article, etc.), or you start being complacent with the code on sensitive matters such as security issues, optimisation, and scalability.
  4. Authors should annotate source code before the review begins.
    • It is important for the author to inform colleagues which files should be reviewed, preventing previously reviewed code from being validated again.
  5. Establish quantifiable goals for code review and capture metrics so you can improve your processes.
    • It is important that the management team has a way of quantifying whether the code review process is effective, such as accounting for the number of bugs reported by the client.
  6. Checklists substantially improve results for both authors and reviewers.
    • What to review? Without a list, each engineer can search for something in particular and leave forgotten other important points.
  7. Verify that defects are actually fixed!
    • It isn’t enough for a reviewer to indicate where the faults are or to suggest improvements. And it’s not a matter of trusting colleagues. It’s important to validate that, in fact, the changes where well implemented.
  8. Managers must foster a good code review culture in which finding defects is viewed positively.
    • It is necessary to avoid the culture of “why you didn’t write it well in the first time?”. It’s important that zero bugs are found in production. The development and revision stage is where they are to be found. It is important to have room for an engineer to make a mistake. Only then can you learn something new.
  9. Beware the “Big Brother” effect.
    • Similar to point 8, but from the engineer’s perspective. It is important to be aware that the suggestions or bugs reported in code reviews are quantifiable. This data should serve the managers to see if the process is working or if an engineer is in particular difficulty. But should never be used for performance evaluations.
  10. The Ego Effect: Do at least some code review, even if you don’t have time to review it all.
    • Knowing that our code will be peer reviewed alerts us to be more cautious in what we write.
  11. Lightweight-style code reviews are efficient, practical, and effective at finding bugs.
    • It’s not necessary to enter in the procedure described by IBM 30 years ago, where 5-10 people would close themselves for periodic meetings with code impressions and scribble each line of code. Using tools like Git, you can participate in the code review process, write and associate comments with specific lines, discuss solutions through asynchronous messages with the author, etc.

I advise you to read the document for a more in-depth view of each point.

It is impossible to refute that the code review will improve the quality of the code produced, reducing the number of bugs and improving the design. There are some additional assets such as sharing knowledge among peers and mentoring more junior team members.

It’s a fact that this process, even if done in a lighter way, will take time to the project. In the review process, we have to add the author’s response time and again the review time again. In an attempt to expedite this consumption of time, I advise that, after two unsuccessful revisions, the author and the reviewer sit side by side and debate the piece of code that is causing the problem.

But it’s really important to state a phrase from IBM’s site –  Slow down to go faster.

I would like to end with a note, which I have as too important. The code review process demands the entire development team to demonstrate high emotional intelligence. As a rule, programmers have a strong ego and feel they are “parents” of the code written by themselves. It’s sometimes not easy to accept less constructive comments. Totally destructive, provocative or jocular commentaries “will do more harm than good,” and may even destroy the morale of a team. It is important to follow these elements, to try to change their attitudes or, as a last resort, to remove them from the team.

Posted on

Test-Driven Development

TDD

Test-Driven Development

The “eXtreme Programming” methodology promotes the test-driven development approach (TDD). What that means?

It is common to see a developer jump headfirst into a functionality. However, TDD advises otherwise.

Short blanket syndrome

You might be wondering “I don’t see what is wrong with that approach”. Let me ask you this question – have you ever implemented/updated/fixed a functionality and by some black-magic, other “way off” functionality broke? I call this the short blanket syndrome. When trying to cover one spot, another is uncovered.

test-driven development diagramXP advises to develop in repetitive short cycles. The developer should write a test and execute the program. The test will fail, we don’t even have any real code to run. Time to write some code, but just enough for the test to pass. If green light occurs, the developer should revisit the running code and refactor if needed.

Write a test, fail. Write code, pass. Refactor.This cycle will continue on and on.

There are many possible tests – unit, integration, functional, acceptance. All of them, as we will find out, are extremely important in the software life cycle.

Unit Tests

This type of testing, as the name implies, should evaluate the smallest unit of code. It should test, typically, every public method and with no interaction with the rest of the system. Basically we will be evaluating the input and output of data to every method. If, in a future update, a developer write some code that will change the necessary input data or the type of output, the test should fail and let the developer know that it may break the entire logic.

It is quite common to find developers using the, so called, “code-driven tests”. That is the same to say that a developer will write the running code and afterwards will write the adapted test.  This inverse process will make your code base really difficult to test, and it is wrong.

When these tests are well written, they may be used as documentation. And this is really interesting.

How many unit tests should a code base have? Well, Robert “Uncle Bob” Martin use to say that 100% of the code base should be covered by tests !!!

Integration Tests

After unit testing, where the small units of code were tested, is time to integrate all the pieces. There are several approaches to this sort of testing – “big-bang”, “top-down”, “bottom-up”, “mixed” and “risky-hardest”. I will let the explanation for those for future posts. However, and if with unit tests one should cover 100% of the code base, integration tests deserves a more concise reflexion. J.B.Rainsberger explains in this talk that we may be creating a small monster. I advise you to watch that presentation (or at least read the article).

Functional Tests

Functional testing belongs to the Quality Assurance (QA) process. Like the name implies, in a functional test we will test a functionality against some specific requirements. The internal process isn’t important, that is why this sort of testing is called a “black-box” test.

Acceptance Tests

We might consider acceptance tests as the last stop before the release. We will write user stories, with the client help, and the test should mimic that story. Let me show you an example:

  • user will access the login screen
  • user will input authentication data (user: admin, pass: password)
  • user will be redirected to screen X

Basically, we will test several possible scenarios, emulating real users. Like I say previously, this is the last phase before the release, therefore QA team and client should be working really close.

Mutation Tests

This sort of testing takes the approach of modifying small pieces (mutations) of your code base. Those mutations should be detected and resolved by your testing suite that would be running in each commit. As you may think, this testing typology is primarily to test your test suite and to help design new tests.

The example provided by Wikipedia, transpilled to PHP:

The mutation might be just transforming the logical condition && to the condition ||.

For PHP we can refer to the Infection framework, and for JavaScript we might choose the Stryker framework.
Following another XP best practice, the entire code base should be continuous integrated, so all these tests must be automated. Worry not, I’m writing another post about this subject.

Have I convinced you that writing code with tests are really beneficial ? Let me know your opinion.