Last Friday, Volexity published a zero day exploit (CVE-2022-26134) on Atlassian Confluence. This post is to share some tips on how to check your Confluence instance is safe, and also some practical advice to protect your Confluence on-prem.
About the vulnerability
This bug affects all versions of Confluence since 1.3.0. It is a critical vulnerability because it allows unauthenticated users to execute code within the Confluence server remotely. According to Imperva Threat Research, there are widespread scanning and attempts of exploitation on the Internet.
How to fix the vulnerability
Atlassian alerted the customers promptly and responded with high priority. We are thankful that Atlassian released the fix in less than 24 hours.
Here are some basic checks that you can execute to check for any traces of malicious attempts. If there is any occurrence, then you may want to engage the security experts for more in-depth foresenic investigation.
URL requests containing ${
Since one of the attack mechanisms is to use ${ in the request URL, it would be helpful to scan the web server access logs for any occurrences. Please update the path of the Apache httpd/ Nginx access logs accordingly.
Based on the Volexity report, there are some IP addresses which are used by the attackers. Similarly, you can grep the access logs to check for any occurrences. Note: It is possible that there may be other attackers using other IP addresses.
Actually, the best form of defense against unauthenticated attacks is to place the server behind the firewall. This will effectively block all attackers from mounting a direct attack remotely. That is a key reason why some security sensitive enterprises are choosing Confluence Data Center. We know that it is not possible for a software to be 100% free of bugs. So there might be another vulnerability waiting to be discovered in the future.
By using Long Term Support release of the product, it reduces the effort to upgrade since the critical security fixes will be available as long it is architecturally possible. This contributes greatly to a quick reaction to any future zero day exploits.
For those organizations who are working remotely, it is possible to access via VPN or use Web application firewalls for added protection. Both CloudFlare and Imperva have announced that their customers are protected from this vulnerability since they will ensure all requests are authenticated before relaying it to Confluence.
Last but not least, do make sure the license technical contacts are up-to-date. As an Atlassian Solution Partner, we have witnessed a number of occurrences when critical alerts from Atlassian are missed due to staff turnover.
One of Jira’s strengths is that it allows 3rd party integration via REST API calls. By providing the username, password and Base URL, it is possible to 3rd party apps to query or update Jira automatically.
In this article, we will share why using API Tokens is a better and safer option than using Password Authentication.
The Power of Passwords
Besides entering passwords on the Jira login screen, it is also possible to provide the passwords on 3rd party applications or scripts to execute REST API calls.
Some of the use cases are like
Create issues from Slack
Send alerts to Microsoft Teams
Update Jira issues with Commits information from GitHub
Integrate with your in-house systems
If the password fails in the wrong hands, it is possible that
Wikileaks of your confidential data
Your Jira system can slow down drastically due to excessive API calls which affects the usage of other users
Benefits of using API Tokens over Passwords
By using API Token, it improves the security of your Jira instance
Safer – The API Token has a certain level of password complexity which defends against dictionary attacks
Isolation – It distributes the risk by having a different API token for each 3rd party integration. It is possible to revoke/reset the token for that application without any impact to other applications.
Differentiation – With a different mechanism, it is possible to apply more stringent checks on the usage of API Tokens (e.g. restriction by IP address range)
Control – It restricts ordinary users from using their credentials to do REST API calls
Availability – For sites running on Single Sign On. Users will not know their passwords other than their Windows passwords
Validity – It is possible to set the expiry date of the token
It allows basic authentication with API Tokens. Currently, it is not possible to use the Jira Data Center’s Personal Access Token together with the username on 3rd party websites.
Warning: If you disable Basic Authentication with passwords in the System Wide settings, you also can’t authenticate on non REST endpoints with API Tokens directly. You can still do that by reusing a session you got from authenticating with an API Token.
Able to limit usage to particular IP ranges
It is possible to limit the usage of the API token to the IP address of the internal system. You can ensure the REST API calls are coming from your trusted network.
Block requests with malicious characters in path
This is a bonus feature which helps to defend against some attack vectors.
Limit usage of API Tokens
It is a security best practice to grant rights only to users who needs it and has proper training. There are incidents arising from users who entered their Jira passwords on 3rd party sites or executed a buggy script.
Tip: We recommend to create a group “jira-api-users” to manage those service account users who can use API Tokens.
Set a validity of the API Token
If the token is for testing or for temporary usage, the Jira admin can just set a shorter validity that will expire automatically. Otherwise it relies on the Jira Admins to remember to revoke the access manually.
Service Accounts typically do not have a password validity. If the service account is from an Active Directory, there could be disruption if there is a 90 day reset password policy.
By using the Active Directory passwords, it is possible that account is locked out of all applications after multiple wrong password attempts.
Control over audit logging
It offers admins a fine level of control over the information to be logged.
This is an ebook on a collection of Jira best practices that Atlassian has gathered from Jira experts around the world.
The title “Be a Jira Hero – A guide for Admins, by Admins” is well deserving. Therefore we recommend all Jira administrators and wannabes to read the 25 pages ebook.
Jira Best Practices ebook
The ebook is well-organised into different sections with short but clear snippets of wisdoms:
How to keep Jira clean and simple for your users
How to use issues effectively
Best practices on managing Custom Fields and Screens
Things to consider when building workflows
Tips on managing backlog in Jira
How to get people to work on the issues
Best practices on designing the Jira dashboards for even more effective reporting
Shortcuts & Hacks to work faster with Jira
How to make use of automation to make life easier
One Common Mistake
For example, the book has highlighted in Tip #20 to incorporate Resolution into the workflows. We observed that a number of Jira admins who did not have formal training tends do not understand the concept of resolution. Consequently, they overlook this important step while creating new workflows. While the Jira appears to be working, the implications are some built-in reports may be inaccurate and an important piece of information is missing.
Our Additional Jira Tip
Dashboards are very useful if they are used correctly. Team members can have visibility on the progress and be reminded on the outstanding tasks. Furthermore, Management can have clear visibility without spending a lot of time preparing routine reports. As a result, users will understand the value of updating their tasks in Jira.
Multiple Filters Statistics Bar Chart Gadget
Horseshoe Gadget Jira
Conclusion
To sum up, if you are a Jira administrator, you should download the ebook and review whether you have been applying the Jira best practices.
This is a “lengthy” guide on the best practices on choosing and managing a Jira app. Coming from our years of experience as an Atlassian Solution Partner, it covers
How to evaluate a Jira app
How to better manage the apps installed in your Jira instance
Things that Jira administrators need to watch out
Jira Apps from Atlassian Marketplace
By installing additional Jira apps, you can add new features to your Jira. For example, some of our popular Jira apps
Being an Atlassian Solution Partner, there are a number of criteria that we consider before recommending a Jira app to a customer. They are:
Fit to Purpose
The app should be able to address the requirements fully. It should be easy to use. The additional feature should not cause confusion to the existing users. We will also assess their design is robust and does not conflict other plugins or use cases.
Atlassian Marketplace Listing
It is very important to check out the app listing on Atlassian Marketplace. It contains a lot of useful informations like
Rating and Reviews
Number of Active Installations
Vendor Accreditation
Compatibility with the latest version of Jira
Support
Rating and Reviews
While the rating and reviews do not mean much for a newly listed app, they do give insight of poor services or badly designed apps. Typically, users will flag out any issues that require attention. So, if there are no negative reviews, it is safe. And if there are a number of good reviews, then it is very safe.
Number of Active Installations
If an app is popular with a significant number of active installations, that is a good sign that the app is useful and functioning without major issues.
Vendor Accreditation
Atlassian has a Top Vendor program who has met the following requirements
App traction – Their paid apps must attain at least 500 active installations
Timely support – Has a Service Level Agreement (SLAs) and committed to provide at support for their users
Support Website – Customers can raise support request easily via a website or email when necessary
Vendor reliability – Atlassian conduct checks periodically to ensure they offer great customer experiences
For those exceptionally good ones, they are further recognised as Gold/Platinum Top Vendor.
Versions compatibility
If the app is only compatible with an older version of Jira, that is a big warning. You do not want to be left dangling helplessly for your next Jira upgrade.
Support
Some apps are explicitly marked as unsupported. Unless you can accept the risks, it is better to avoid them.
Data Center Approved
Data Center approved apps are those that have passed Atlassian’s stringent criteria for use in Jira Data Center environments. As it takes considerable effort to become a Data Center Approved App. vendors will usually do that for their established apps. Therefore this is a plus point.
Thorough Documentation
Although some users will usually dive in without reading the user guide, it helps when the documentation contains
A quick overview of the functionalities
Comprehensive writeup and how to use
FAQs to facilitate on-boarding and best practices
Up-to-date release notes
An easy to navigate structure
Unless the app is really simple or open source, it is a risk to install an app with a one pager documentation.
Pricing
Last but not least, the pricing of the app is also an important consideration. The app should have a reasonable pricing so that you will be assured of support for long term. It is also possible that you will upgrade to higher user tier or Data Center as Jira becomes an enterprise application. Hence the pricing for higher tiers should not exceed your budget.
For free apps, they should be from a credible publisher with significant number of installations. Otherwise, the publisher may withdraw support in near future.
Before Installing an App
Downloading the compatible version
There is a common mistake for Jira administrators who are unable to download the app via the Atlassian Universal Plugin Manager. The default download link on the Marketplace listing is for the latest version of Jira.
If you are not using the latest version of Jira, the correct steps to install/upgrade the app will be
Click on the Versions tab
Select See all … versions
Hover on the topmost version which matches your Jira version
Click on the Download link
Upload the file via Atlassian Universal Plugin Manager in Jira
Testing
It is a best practice to install the Jira app on a Test environment first. It is not advisable to install an app directly onto the Production environment. This is because some apps may make changes to the database. Some changes are not be reversible when you uninstall the app later.
Maintaining a Change Log for the Apps
We usually advise our customers to create a Confluence page to track on the app installations, upgrades and uninstallation. This best practice helps to keep all the related information organised neatly on a single page.
The change log table can include the following information
Date of Change
Changed By
Name of App
Action Performed (e.g. install/upgrade/uninstall)
From Version
To Version
Jira Issue Key
Additional Notes
This will be useful especially when there are multiple Jira administrators who can perform changes. Although the audit log has a record of the changes, the default data retention period is only 90 days. It is also not that easier to read.
By using a Change Log, the administrator can link the change with a Jira issue and add notes. The Jira request helps to identify the user who raised the request to install the app together with the background context.
When there is an incident, the App Change Log will be useful to identify issues caused by app changes.
Things to watch out
It is not the end of the story after installing the app. There are a few useful tips for Jira administrators to watch out.
Hardware Sizing
With more apps installed, it is likely that the memory footprint might increase over time. It will a best practice to check on the JVM memory utilisation from the System Information page after people start using the features. Likewise, CPU usage monitoring is also a best practice.
Expiring Licenses
There are a handful of apps which will stop working without an active license. So be sure to renew your licenses before they expire to gain access to support and newer releases.
Being informed
You can click on the Watch app link on the Marketplace listing. By doing so, Atlassian Marketplace will send you an email whenever there is a new version released.
Upgrading the Apps
As a best practice, it is important to upgrade regularly to get newer features, performance improvements and bug fixes. For security vulnerabilities, you should upgrade as soon as possible once you received a security advisory.
Scheduling Upgrades
Although there is no need to schedule a maintenance downtime for installing/upgrading Jira apps, it is a best practice to do that during off-peak periods. During an app installation, there is a short interval where the Atlassian Universal Plugin Manager removes the old version and installs the newer version. You do not want any critical operations in between that period.
Also some apps may execute jobs during the upgrade to do some data crunching to support a new feature. Therefore, please read the release/upgrade notes for the app before you click on the upgrade button in Jira.
