Cybersecurity

Certificate transparency and how cybersecurity teams can use it?

Certificate Transparency (CT) and how knowing about newly registered digital certificates can put your cybersecurity team into proactive mode.

Jonathan Evans

Jun 4, 2024 • 7 min read

Source: certificate.transparency.dev

Imagine if you could see attackers building systems to attack you! Imagine being able to put protection in place before the phish are sent! Imagine being able to identify systems that have been forgotten about, not maintained and vulnerable on the internet! In this blog we will be sharing knowledge about Certificate Transparency (CT) and how knowing about newly registered digital certificates can put your cybersecurity team into proactive mode.

The background to certificate transparency

CT is the publication of details on all new digital certificates being issued by the root certificate authorities. It's covered by IEFT RFC 9162, backed by some of the biggest tech firms on the planet, been around for over 10 years but it’s surprising how many people haven’t heard of it.

Root certificate authorities are the national or international organisations that are responsible for issuing new digital certificates. The certificates for all root certificates are built-in to your computer and allow your browser to know who to trust.

The publication of the new digital certificate registrations means that there is now a single common ledger that allows inspection and greater oversight. If a root certificate becomes compromised or certificates are issued in error they can be quickly identified, traced and revoked.

“Digital Transparency is an ecosystem that makes the issuance of website certificates transparent and verifiable”.

CT allows browsers (Chrome, Brave and Safari) to inspect the certificate registrations prior to allowing end users to have access to web sites. This provides an extra level of validation and protection. We view it as holding the root certificate authorities to account because the CT logs means everything is recorded in real time and cannot be altered.

Where can I see the root certification authorities certificates?

If you have Windows 10 or 11 running the certmgr.msc command will give you the following console:

A graphic showing the certificate manager console directories — Certificate manager console on Windows 11

Selecting Trusted Root Certification Authorities gives you a very long list of CAs that your computer implicitly trusts. Below is an extract and not the full list.

A partial list showing 20 plus root certificates — List of certificates found under Trusted Root Certification Authorities

Who supports certificate transparency?

As you can see above it has the backing of some of the largest tech firm in the world!

A list of supports including apple, google, facebook, digicert, lets encrypt and 3 others — Certificate transparency supporters

What can I do with the certificate transparency logs?

Well, the first and easiest thing to is simply use a free service to identify your domain and all the associated certificates. This may help you locate some deprecated certificates or sites that you’ve forgotten about and are highly vulnerable to compromise.

There are a number of “Certificate Transparency monitors”. Here are two that provide domain search services for free. Simply type in your domain and they will perform the search for you. There is no pre-registration required.

Entrust Certificate Search - Entrust, Inc.

Screenshot from the Entrust search tool with a box for entering your domain name — Entrust CT search tool

crt.sh | Certificate Search

How can I use CT to defend against phishing attacks?

So far so simple. This is where things get a little more interesting and it will be worth the effort. There are various ways the real-time digital certificate logs can be inspected and one such mechanism is called Certstream. Think of it as a constant stream of information that you can tap into.

The official website can be found here - Certstream (calidog.io).

You can find out more details about Certstream on github here - CaliDog/certstream-python: Python library for connecting to CertStream (github.com)

Once you have downloaded the repository from Github you will need to:

1) Build it – do this by running step-up.py with the build switch

2) Install it – do this by running set-up.py with the install switch

3) Run – do this by running the echo.py script

Once it’s running you will see that the world is very busy at registering certificates at a surprising rate. This means that you will quickly be over-run with information.

How to customise it and make it more relevant

We have recently been having some fun around refreshing our technical skills. One area that has really delighted us is AI and specifically when applied to python programming. Sometimes the subject can be a little frustrating, you know what you want to do, and you know the steps to make it “happen” but you don’t know the low level precise details required. Well, that’s where AI excels.

We asked AI to customise the code and make it more relevant to cybersecurity. The resulting code is below and as you might be able to work out. It is focused on the brands or domains we are interested in (listed in the file called targets.txt) and then looks for new certificates where the domains have been created in the last 30 days it also looks for some suspicious words that may be of interest to attackers – login, verify, secure, update, banking and account.

Customised version of echo.py

import logging

import sys

import datetime

import certstream

import re

import whois

from datetime import datetime as dt

# Read target strings from the file 'targets.txt'

with open('targets.txt', 'r') as f:

target_strings = [line.strip() for line in f]

# Define suspicious keywords

suspicious_keywords = ['login', 'verify', 'secure', 'update', 'banking', 'account']

def is_phishing(domain):

# Check for suspicious keywords

for keyword in suspicious_keywords:

if keyword in domain:

return True

# Check for excessive hyphens or numbers

if len(re.findall(r'\d', domain)) > 5 or domain.count('-') > 3:

return True

# Check domain age

try:

domain_info = whois.whois(domain)

creation_date = domain_info.creation_date

if isinstance(creation_date, list):

creation_date = creation_date[0]

if creation_date:

age = (dt.now() - creation_date).days

if age < 30: # Consider domains younger than 30 days as suspicious

return True

except Exception as e:

logging.error(f"WHOIS lookup failed for domain {domain}: {e}")

return False

def print_callback(message, context):

logging.debug("Message -> {}".format(message))

if message['message_type'] == "heartbeat":

return

if message['message_type'] == "certificate_update":

all_domains = message['data']['leaf_cert']['all_domains']

if len(all_domains) == 0:

domain = "NULL"

else:

domain = all_domains[0]

# Check if any target string is in any of the domains

if any(target in d for d in all_domains for target in target_strings):

for d in all_domains:

if is_phishing(d):

sys.stdout.write(u"[{}] {} (SAN: {})\n".format(

datetime.datetime.now().strftime('%m/%d/%y %H:%M:%S'),

", ".join(all_domains[1:])

))

sys.stdout.flush()

logging.basicConfig(format='[%(levelname)s:%(name)s] %(asctime)s - %(message)s', level=logging.INFO)

certstream.listen_for_events(print_callback, url='wss://certstream.calidog.io/')

original source: CaliDog/certstream

How to exploit this freshly gained threat intelligence?

If you use this for your company domain then you should be able to cross reference the output with your change control or release management logs to identify domains that are not owned by you and could be used to trick your staff or clients. Once you have this information you could carry out the following proactive steps:

Applying blocks on your web proxies and your email gateway.
Consider issuing a domain take down order.
Instruct your third-party providers to apply the same blocks to prevent impersonation of your brand.

I hope you’ve enjoyed this blog. It’s a little more technical than usual but it gives you an idea of how you can get one step ahead of the attackers!

How IT Security Locksmith can help

If your IT Security or Cybersecurity team are struggling or they simply need some help why not get in touch to provide them with some additional skills and capabilities. We offer:

Targeted engagements to focus on you areas of greatest concern
Team mentoring or support for junior and senior staff from an experience cybersecurity professional
Cybersecurity reviews and recommendations for key processes to improve them further
Post incident recovery assistance designed to focus on key areas of detection, protection and incident response planning