We’ve just published a new slide deck on SlideShare titled “Technical Debt 101” (click the link to view, or watch embedded below).
The slide deck was written by Intechnica Technical Director, Andy Still. Make sure you take a look at Andy’s regular blog posts, including expanded posts on Technical Debt management, over at InternetPerformanceExpert.com.
Andy explains why Technical Debt doesn’t have to be negative, but it does have to be carefully managed. These slides give a quick run-down of best practice to approaching Technical Debt management.
We have extensive experience in managing Technical Debt. Get in touch to talk to us about managing your Technical Debt the right way.
Following on from my post two weeks ago about specification by example and application maturity, this piece is about requirements traceability.
Software development processes have traditionally worked from signed artefacts and signing up to agreed pieces of work. This would normally describe a set amount of development time, some testing time and some support time to resolve defects and help bed-in the new application. An important part of this process is a description of what will be delivered. This is a key document in the specification by example process. It is important when creating the original statement of requirements and describing the user stories in tools such as Speclog that all requirements have been captured and that there is a common understanding between the project sponsors and the suppliers of how this functionality will be used. By describing key business flows in terms of the behaviour of the application an opportunity is provided for areas of discussion and the understanding of the supplier and the customer can be explored.
These activities drive the test approach. Traditional testing analysis would follow on from here. Often this would be long weeks of Business Analysts creating specification documents and from there test analysts would follow the same path and develop test packs which examine and validate these requirements. In the mean-time developers will work with the documents and the outline discussions, and begin their development approach. In the old days this would have created detail specification of requirements, a detailed development approach and detailed test approach – all of which would have to be approved and signed off. This produces a lot of documents and the big problem with them is that it assumes the requirements are fully understood at the beginning of the process. Any changes are managed through change requests, which require impacting and updating of all these documents. It is not a flexible process but although slow, delivers good quality software.
In the same way that Agile has helped to remove all the unnecessary documentation in development and testing of new applications, specification by example is looking to broaden/develop this concept to address the requirements itself. In the area of Quality Assurance the big challenge with ever more complex systems is measuring what areas of the requirements are being met by which development activities and then validated by which testing activities. In its broadest sense specification by example attempts to address this by a straight-forward approach that states that tests will be developed for each requirement specified in the user story.
The challenge is what artefacts we need to create and how they fit in with the user stories. For instance can/will test approach documents be taken straight from the stories that are created in the analysis. The examples and systems that have been discussed in Agile conferences and across the web have been small with simple business processes which have limited number of functional combinations of application modules. And these systems have been created from scratch or are small improvements. This work hasn’t taken place with existing systems that may not have full documentation or where new software and changes to existing applications take place.
This is where testing theory and previous processes can supplement the given process. It is important that the relationships between the requirements and stories are understood. What are the most important part of the applications ? Which are the most complex? By using the traditional requirements traceability it is possible to create a an application relationship map – which then can be used to drive the test plan and more importantly be used to manage the delivery of the application functionality. This will help with deciding the critical path and the main release points where key modules from a testable business process.
Where defects are being seen then it can steer back to the stories and allow us to understand whether there is something missing from the requirements or something has changed or details need to be more fully understood. This takes us back to the largest flaw in current development/test methodology. When the initial analysis takes place assumptions are made using the business/test/developer analyst experience and understanding of the business. The won’t necessarily align and more importantly they will change as more work is done. It is vital that all that information is shared – and that is at the heart of collaborative processes such as specification by example try to engender. But at the moment the process seeks to implement a one-size-fits-all approach and only looks at basing everything on the user stories. There may well need to be additional process activities.
What we want to find out over the next few months is how requirements traceability works in the Specification by Example area. It has proved to be a valuable tool in Agile projects – where there isn’t the time to record all the details of all the requirements and in Waterfall where the long time periods of development and testing activity can be managed through the matrix. It has weaknesses – it doesn’t work well across a large number of conflicting requirements and doesn’t work well when the business capability being developed splits into small components. But it will be interesting to see what it produces and future blogs we will be recording what we see.
It is often thought that Testing/Quality Assurance is just about finding defects – or as some people think – playing with the software and seeing what works and what doesn’t. As usual the reality is a lot more involved. The finding of defects is only one aspect of what QA does. When preparing a test approach you need to ask some fundamental questions that appear to have obvious answers, but when you think about them a bit more – you are not so sure.
- How much testing needs to be carried out before the application is ready to deploy?
- What quantity of defects found in a test phase demonstrates that the application meets customers’ acceptance criteria?
- What business activities need to be evaluated to prove that the application will be able to carry out business tasks as required?
The third question is actually the key. Any project will have a set of opportunities or a capability that a business needs. They will measure the benefits of employing people to create code and time to deliver the project by the value benefits these capabilities add to the profits of the company. This could be from consolidating a number of accounting systems, being able to sell via an electronic market place a new service or being able to manage more effectively the routes that delivery drivers take to deliver stock. All add value to the bottom line. To achieve this the project has to be very clear about what new software or changes to existing applications in a language which there can be no ambiguity. Working with the developers, Quality Assurance exercise the software and measure whether the code meets these needs. And this is done through looking at the behaviour of the application rather than trying to take apart all the constituent parts. Because specification-by-example builds testing into the development process – the lower level code modules will be tested and validated with the requirements. So low level noise (syntax errors, validation errors etc.) shouldn’t be present. So the QA focus when using the application will look at validating the user-stories and from that examining the requirements. So how many defects found in a test phase mean that the software is acceptable for use? The key measure used by QA is to look at the spread of defects by functional area. Traditionally these areas will have been mapped in the early part of the project via a traceability document. This document measures the business criticality and complexity of the requirements and relates that to use stories on how the new functionality will be used. The challenge is going to be that in an Agile context, making it lightweight enough so that enough information is provided but it doesn’t take a long time to produce. The drivers behind the Agile methodology emphasise only doing those tasks that are necessary – to drive the release forward and understanding what tasks need to be done on the critical path to deliver a release. Traditionally the readiness of an application for service was proved by the level of defects seen in each functional area. Traditional QA activities come from a waterfall past where specific quality gates must be achieved and everything moves in slow sequence. In the new methodology it is necessary to only do the minimum amount of testing that proves that project requirements can be met and that the application will perform provide a reliable and effective level of service. Without the old techniques of testing all parts of the application the focus moves to testing those areas that are critical to the purpose of the change and support the business capability being delivered. So it is possible to use the old technique but it is necessary to put it in a new context making the process lighter and more flexible. This is just the start of some big changes in software development methodologies. When defects are seen they will have much higher impact and could have slipped unnoticed through a regression change or integration point where two modules have been brought together for the first time. The new processes will need to give this information context so that critical issues are fixed quickly without impacting overall delivery times. Testing will focus more on end-to-end process flows (business stories) and less on small components such as data validation and software branch variation. This will not mean that the testing won’t be done, but it will be done at an earlier point. This is only one area where “Specification by Example” will make subtle – but fundamental changes to application delivery. Important Agile behaviours such as “fix-first-time” – “active collaboration” will need to be enforced to drive through quality. But old measures such as defect latency, burn rate on feature delivery and making decisions on what will be a realistic time for launch of the application are going to be equally important. These are going to be exciting times for project methodologies and it will be interesting to see where we are in 6 months time.
I’m in the final stages of preparing my presentation and workshop session for the UK Test Management Summit next week in London and its making me think more about cloud computing in general as well as performance testing. Either testing in cloud environments or using the cloud to deliver more scalable performance tests.
Intechnica’s research paper last year, entitled “How Fast Is The Cloud?” investigated the relative performance of a simple eCommerce application on various different cloud platforms including IaaS and PaaS options. We demonstrated that a well implemented cloud solution could out-perform traditional hardware but that poor implementations would confirm cloud-sceptics suspicions about poor performance in the cloud.
At Intechnica, as well as using cloud environments to functionally and performance test code that we’re developing for clients, we use cloud based performance test tools to test our customer’s own test environments. By using cloud based load generators (injectors) and the Intechnica TrafficSpike product, we can quickly provision tens of load generators, use them for a few hours and then decommission the servers. This allows for highly scalable, comparatively low cost performance testing particularly when compared to trraditional models where multiple servers sit idle waiting for the one day per week or month where they’re used to their full potential.
The trend in performance testing seems to be a move away from traditional performance test tools and towards cloud-based load generation. This is demonstrated by the growth in companies such as SOASTA, LoadStorm, blitz.io and BlazeMeter. Our workshop at TMF will give test managers the opportunity to discuss these different test technologies and obtain a better understanding of cloud performance and the implications for their business. As well as this we’ll be giving attendees the opportunity to use Intechnica’s Cloudflex product to see how easy it can be to provision multiple, identical test environments for themselves.
I’m looking forward to meeting attendees next week to discuss the implications of cloud computing for those of us in the testing industry.
Intechnica’s Head of Performance, Ian Molyneaux recently wrote an article for silicon.com in which he said, “The benefits of cloud computing are there for the taking – but only if you have the right skills to exploit them”.
Ian is the Head of Performance at Intechnica
We all know that in the world of Information Technology, different skill sets are in demand at different times. Cloud computing is currently “flavour of the month” with Gartner predicting that by 2015 half of CIOs expect to be operating their applications and infrastructure from the cloud.
To support this shift from old fashioned “tin” to a virtual, cloud-based infrastructure, technologists need to brush up on a number of key skills. There are numerous benefits in moving to the cloud such as reduced cost, rapid deployments and on-demand access to a large pool computing resources. Despite the obvious benefits there are a number of key skills which are critical to successful cloud deployments.
These skills are:
Application development and management. Applications needs to be designed specifically to exploit the potential of the cloud, they need to be monitored throughout the application lifecycle to allow updates and improvements to be incorporated into subsequent releases.
Security awareness is paramount, as data communications move out of the data centre and into the public domain.
Deployment should be automated otherwise you’re missing out on flexibility, one of the key benefits of the cloud.
The cloud is full of commercial opportunity but it is not as simple as many expect. IT departments need to look carefully at their teams to consider who in their talent pool are self-learners, who have transferable skills and who are unlikely to make the change.
Read the full article on silicon.com.