https://confluence.atlassian.com/bamboo/bamboo-best-practice-388401018.html
In this section
https://confluence.atlassian.com/bamboo/bamboo-best-practice-sharing-artifacts-402033116.html
General overview
We've already had a look at techniques such as 'fail fast' and 'artifact promotion' as ways of improving your Bamboo processes in the using stages Best Practice guide, but here we're going to dig a little deeper and look at some ways that you can get artifact sharing to work for you.
Best practice approaches
Sharing build artifacts with downstream processes
See Artifact promotion for a description of this technique.
How do I configure artifact sharing between jobs?
In Bamboo, artifact sharing between jobs is configured using the Artifacts tab on the plan's configuration:
Check out Sharing artifacts between jobs to learn how to configure your Bamboo server to take advantage of artifact sharing between jobs.
Sharing artifacts between plans
Objective
Identify and describe how artifact sharing between plans can be achieved
Learning Outcomes
After completing this section, you will understand how to share an artifact between plans
Overview
We discussed above how we can achieve significant time benefits from capturing and sharing artifacts to downstream processes rather than checking out and compiling each time the artifact is required. Generating an artifact at the top of the development pipelie, and passing it to successive downstream processes also has the benefit of ensuring the integrity of the code is maintained throughout the pipeline, because we know it is the exact same code that we tested earlier on. We also discussed how we can manage passing artifacts within a build plan, but let's suppose that we want to pass artifacts between two plans? Easy: We use the download artifact task to make the artifact available from one plan to another.
Example scenario
Let's consider the following artifact sharing example:
Imagine that we have a build plan that creates and uses an artifact - Artifact A. Now let's suppose that we also have a child plan, and we would like to use Artifact A in this plan for some other purpose. Bamboo doesn't technically allow you to share artifacts between plans (but watch this space), so we can use a work around to get our artifact shared into the child plan: We copy it from the parent plan to a remote storage location, then use the artifact download task to obtain it for the new plan. Note: This approach differs significantly to the process for sharing artifacts between jobs.
Parent Plan
Step 1: Checkout & compile - We need to check out the relevant code from the repository and compile it into an artifact. Our artifact is now defined and available for use by downstream jobs. Let's give it a name - Artifact A - and specify its location, so downstream jobs can find it, though of course only jobs in downstream stages can consume it
Step 2: Testing - We can use some Fail Fast methodology and run some tests on our artifact before we go any further. We can conduct short and rapid unit tests and longer functional testing on our artifact. But we already know that artifact sharing can be used to increase testing speed in both cases
Step 3: Deployment - When testing is complete, the artifact can be deployed to a QA environment by a consuming job that runs a deployment script against it, but we still need to share it with the child plan
Step 4: Copy artifact out - The final step of this plan is to use a task to copy the artifact out to a remote location such as Nexus or Artifactory, using the applicable Bamboo plugins. Alternatively, simply run a script task to copy the artifact to a remote file server location on your own network
Child Plan
Step 1: Copy artifact in - The first step of the child plan is to use a task to copy the artifact in from where the parent plan left it. Depending on the method you used to copy it out, you may require a task that utilizes a Bamboo plugin.
Step 2: Business as usual - Now that we have copied the artifact in, we can perform regular Bamboo operations as part of an ongoing build plan. These could be additional tests, or deployments into different environments
Extending artifact sharing
And of course, once we have our artifact neatly stored in remote storage, the artifact download task means that it can associated with any build plans that we may want to run.
Conclusion
Artifact sharing is a powerful technique for making single artifacts available. Artifact sharing across plans allows us to make artifacts available for different build plans from one checkout and compile. We know that we will always be using a consistent artifact which reduces the time overhead of multiple checkout and compile steps.
How do I configure artifact sharing between jobs?
Artifact sharing between jobs is configured using the artifact downloader task:
Check out Sharing artifacts between build plans to learn how to configure your Bamboo server to take advantage of artifact sharing between plans.
Source: https://community.atlassian.com/t5/Bamboo-questions/Plans-Stages-Jobs-best-practices/qaq-p/395270
Here are some general guidelines for structuring your Stages in Bamboo:
Fail Fast
The number one goal should be to structure your Stages in order of speed. The earlier you can find problems, the better it is. Move functional tests that take a long time to run into later Stages, or break them up and paraellise them to shorten the overall test duration.
Stage 1: Compile – Depending on your technology, you will have to compile your application first. If the compilation doesn't work, we are done and don't even need to kick of other Jobs.
Stage2 : Unit Tests – When something is broken, you want to find out as soon as possible. Your unit tests are the first line of defence. Ideally you have a lot of them. They run fast and catch a lot of problems within a few minutes.
Stage 3: Functional Tests – Functional Tests allow you to test the application with all it's dependencies, which is great. Unfortunately tests also take a longer to run.
Stage 4: Deployment – The deployment comes last. You might have some QA-Deployment Stage before the actual production deployment to perform some smoke tests or manual QA.
Don't Waste Resources
The great thing about failing fast is that we can detect potential problems early and don't need to execute the rest of the build. If we take the above example, when a test fails in the Unit Test Stage, we won't even bother running the more expensive functional tests... we already know that they are likely to break and are better of fixing the unit tests first.
Test the Artifact You Want to Deploy
Again, depending on your language, you are likely to generate an artifact of your application. You want to make sure that you only have to build this artifact once and test the same artifact in other stages before it gets deployed. You can either pass it via Bamboo's artifact sharing feature to subsequent Stages or use an external artifact management tool. That's completely up to you and depends on what existing tools you already use.
Jobs vs. Tasks
You can have multiple Jobs within a single Stage. The great thing about Jobs is that within a Stage they can be executed in parallel. Which means that you can run a bunch of Tests in parellel or deploy to different servers in parellel. Since Jobs run in parellel it's likely that they are being executed on separate agents and potentially completely separate physical machines. Hence you don't want any dependecies between Jobs like relying on the fact that they all access the same working directory.
Tasks within a Job on the other hand are always executed in serial on the same agent. Hence you could create a Task that assumes a certain state created by a previews Task. For example you might have a Task that modifies some property files before the tests are being executed with the next Task.
I hope this helped a little. Expect more blog posts and documentation on Stages and Continuous Delivery over the next couple of month.
Last but but not least, the following blog post is a rather simple example on how one could structure their Bamboo Stages:
Plan your regression testing strategy by asking the relevant questions
When designing your test strategy, it is important to consider regression tests as part of your overall test plan.
Regression testing, as you probably know confirms that new code changes don’t impact the existing code/functionality. This is an important parameter to control the quality of an application under test, especially with frequent code changes.
Regression testing allows Agile teams to manage risks related to frequently changing requirements and code changes.
Testers implement regression testing in various ways. Either they repeat the exact same tests that were carried out before the code changes occurred. Or they reuse the older data, use different data and some additional conditions to carry out these tests.
One important distinction that helps you understand regression testing better is that there are two types of regression tests:
- Function regression tests
- Non-functional regression tests
Functional regression tests are those that make sure that the software is working as intended.
Non-functional regression tests confirm if the changes have an impact on performance or security issues.
Ideally, the regression test plan should include test automation. This is because regression tests are repeated frequently – with every code change. This means they need to be executed constantly.
Automation of regression tests gives a quick evaluation of the software’s functionality.
This, in turn, helps testers to look at the edge cases using exploratory testing. So, QA teams must find a good balance of both manual and automated tests in their regression test plan and use the relevant tools for each.
Obviously, tools play an important role in the execution and management of regression tests. Right from full-featured vendor products to open source options, there are several regression testing tools that are used in the market.
Some popular choices for Regression Testing include:
- Selenium
- Ranorex Studio
- Katalon Studio
- Sahi Pro
- Watir
- SmartBear TestComplete
- IBM Rational Functional Tester
How to define your regression testing strategy
When formulating your test plan there are many factors that you need to look at. Here are some questions to help you define the right regression testing plan:
1. How much coverage do you potentially need?
This depends ultimately on your testing goals. Define the goals and coverage requirements as clearly as possible at the outset. If there is confusion around the goals of regression testing, then the tests become inefficient and expensive.
Often a single test takes care of multiple risks or areas of application. This is why coverage is hard to track during regression testing. Use a test management tool that enables you to track coverage and provides quality metrics. And understand coverage to communicate how regression tests impact other project members.
2. How will you use test automation within regression testing?
It is important to think of the techniques you will use to maintain and execute the tests. For example, how to apply test automation within your plan or selecting tests for manual testing and automated testing.
Deciding what skills are required, the areas of application and the right toolset will help you refine your regression testing plan. It will also allow you to plan for test automation in parallel with your continuous integration pipeline.
3. Defining the environments
What environments are required to execute the regression tests? Think of the data you need to carry out the tests and custom configurations if any to be deployed. Remember, that some tests might need to be executed against different configurations of the program.
4. Reporting
Reporting is crucial to any type of test plan. Reporting the status of testing including the level of detail, bottlenecks and priorities helps you plan better. With effective reporting you can leverage the following benefits:
- Traceability matrix
- Dependencies between test cases and requirements
- Change reports of previous and current releases
- Risk-based analysis, output from code coverage
- Eliminating unnecessary tests by pruning out test cases that are no longer required.
Since regression testing includes several types of tests there is no ‘ideal’ formula to do it.
As the application scope and code complexity grows, there are multiple and frequent patches to the system. In such a case, you need to select the optimum tests to save cost and time. Answering some of the above-mentioned questions will help you choose wisely and deliver a quality product.
Although we have finalized the four questions that are required to define the regression testing strategy, the main question which still remains unanswered is how can I implement these parameters? One of the idealistic answer is to have test management app do it for you.
A Test Management App such as QTM4J allows you to define, create, track and plan your testing activities inside Jira. With the test management app, your agile testing team can plan the testing activities across sprints/releases, as well as monitor the test executions with test cycles. The app also enables test automation by allowing integration with automation tools such as Selenium, TestNG, QAF/QAS, JUnit, etc.
Using a test management app inside Jira, you can configure environments and platforms against which testing needs to be performed ensuring fast and easy capturing of the execution results. Also, Test Management App such as QTM4J which provides extensive testing analytics with 30+ reports, enabling you to know your product readiness at any given point in time
- Log in to post comments
1 comment