The Golden Bless or how I learned to bypass VPN

During my gym session, I ran across a hard problem. For SSH access, how could one replicate CloudFlare Access and not require VPN to ssh into Production assets?

What is Cloudflare’s Access?

Looking at CloudFlare’s Access, all they did was copy Bless, turned it into a service, and made Cloudflare’s edge a jumphost.  It is still early days but works really well and is coming along.


What is Bless?

“…SSH Certificates are an excellent way to authorize users to access a particular SSH host, as they can be restricted for a single use case, and can be short lived. Instead of managing the authorized_keys of a host, or controlling who has access to SSH Private Keys, hosts just need to be configured to trust an SSH CA.

BLESS is an SSH Certificate Authority that runs as an AWS Lambda function and is used to sign SSH public keys….”

How would this work?

Traditionally, a well-architected AWS account would have users (and their associated credentials) tied to MFA. Now one could move to aws-okta (or okta-awscli : which gets role credentials via okta aws assume role.) The addition to make Bless work is that the AWS role(s) have a policy that allows operators access to a kms key, which has a policy that allows encrypt based on their user (which is an IAM variable.)  The role also has access to lambda, which generates the ssh cert. The bless client generates a kmsauth token, using encrypt on the kms key. It takes that token, then calls the lambda, asking for a cert, passing in the token and their username. The lambda verifies the token, which maps it to user. it uses that info to generate a cert specific to the user

Common Questions:

It’s a little fuzzy from just looking at Bless how RBAC comes into play. Does the user cert include group info? Or do you have to add users outlined in the certs into groups on the machine as part of the CA cert trust process?

It helps that all nodes have info on all users and limit access to groups.  

That seems… complex. Like every time you get a new engineer who needs SSH access to Production assets, there’s a puppet/salt/ansible/kill and fill that updates with new user info?

Pre-generate nsscache files, deliver those cache files to all hosts, and point /etc/passwd.cache, /etc/shadow.cache, (and others) to the downloaded files.  I used to install the users via salt, but after ~500 users it started becoming unmanageable. The .cache versions of the files are the exact same format as the non-.cache versions.  You're really just shipping around passwd, shadow, and group files, but in a way that won't trash your systems if one buggers the system. Nsscache populates the data for libnss-cache.  Since you are distributing the data, one doesn’t need nsscache on every system.  

Where may I find additional information about Lyft’s Bless fork?  

Additional information may be found @ https://eng.lyft.com/blessing-your-ssh-at-lyft-a1b38f81629d .  That was Lyft’s first implementation and released their version of the Bless stack.  


Example Tools

https://github.com/Netflix/bless

https://github.com/lyft/bless (another fork)

https://github.com/chanzuckerberg/blessclient

https://github.com/ejcx/sshcert

https://github.com/uber/pam-ussh (don’t try it for anything other than sudo.  Thar be dragons.)

 https://github.com/google/nsscache (only really using  https://github.com/google/libnss-cache )

What is a modern, dynamic service and its' building blocks?

When I look at the Cloud Native ecosystem, I am astonished.  The vendor space’s market capitalization is near $7.78 trillion with funding of $12.26 billion.  Below is a rough ecosystem image.

 

As I work through the ecosystem, there is no evident, leading “best practice. Within modern, dynamic environments, there are not obvious answers to empower an organization to build and operate scalable applications.  There are no best practices to enable loosely coupled, resilient, manageable, and observable systems.  Much less do not require 5 FTEs to enable repeatable, predictable, high-impact outcomes. 

 Below are high level challenges one will solve as they build and run their modern, dynamic service:

Containerization

  • Commonly seen with Docker

  • Any size and dependencies may be containered

  • Eventual microservices architecture


Registries and Runtime Execution

  • Harbor.io is a great registry which stores, signs, and scans container content.  When hooked into Clair, may provide vulnerability information

  • If docker isn’t your thing, OCI-compliant containerd, rkt, and CRI-O work great


Distribution

  • An implementation of the Update Framework, Notary, is great to start off distributing your services


CI / CD

  • Changes to the source code automatically result in new containers built, tested, and deployed

  • Hopefully canary with blue / green deployments

  • Automated deployments, rollbacks, and testing

  • Orchestration and Application Definitions

  • Kubernetes is leading the orchestration market

  • Aim to utilize a certified Kubernetes distribution or hosting platform

  • Helm charts enables one to define, install, and upgrade even complex Kubernetes enabled services


Analysis and Observability

  • Find the right services for logging, tracing, and monitoring

  • Prometheus is great for monitoring

  • Fluentd is wonderful for logging

  • Jaeger isn’t bad at tracing.  Otherwise, look for an Open-Tracing compatible solution


Discovery, Mesh, and Proxy

  • CoreDNS is flexible and fast.  Great for service discovery

  • Linkerd and Envoy enable mesh architectures, health checking, routing, and load balancing


Networking Policy Enforcement

  • Istio, Flannel, Calico, or Weave Net are decent general purpose network policy engines

  • Uses range from authorization and admission to data filtering


Database and Storage

  • It really depends on the storage type

  • If one is utilizing MySQL, Vitess works to scale and shard

  • Rook works as a storage orchestrator

  • Etcd provides mechanisms to store data across clusters

  • TiKV works well as a highly performant transactional key-value store


Streaming and Messaging

  • When JSON-REST is not enough, gRPC or NATS is the way to go. 

  • Generic RPC usage is implemented in gRPC

  • Complex messaging utilizes NATS (pubsubhub / subscriptions, request / reply, load balancing, etc..)

 

 

A collection of Bug Bounty polices and statements to run a program

Recently, a friend reached out to me for a simple Bug Bounty vulnerability disclosure policy. While combing through my vulnerability disclosure polices library, I realized there wasn’t an acceptable policy to be found.

The default policies for the popular bug bounty platforms leave much to be desired or are wrapped in legalese not easily understood by those whom English is a second / third language.

Below is a simple policy which can be applied to ones’ bug bounty program. Enjoy!

Vulnerability Disclosure Policy

INSERTENTITYNAME is committed to treating our customers’ data with the utmost care. As part of this, we encourage security researchers to put our security to the test - and we offer a variety of rewards for doing so. We look forward to continuing to work with the community as we add new features and services.

Program Rules

  • Automated testing is not permitted.
  • Follow INSERTBUGBOUNTYPLATFORMNAME’s Disclosure Guidelines - URL TO BUG BOUNTY PLATFORM DISCLOSURE GUIDELINES
  • Test only with your own account(s) when investigating bugs, and do not interact with other accounts without the consent of their owners.
  • You must be the first person to report the issue to us. We will review duplicate bugs to see if they provide additional information, but otherwise only reward the first reporter.
  • We award bounties at time of fix, and will keep you posted as we work to resolve them.
  • Contacting our support team ([email protected]) about the status of a INSERTBUGBOUNTYPLATFORMNAME report will result in an immediate disqualification for a bounty for that report.

Exclusions

The following bugs are unlikely to be eligible for a bounty:

  • Issues found through automated testing
  • "Scanner output" or scanner-generated reports
  • Publicly-released bugs in internet software within 3 days of their disclosure
  • "Advisory" or "Informational" reports that do not include any INSERTENTITYNAME-specific testing or context
  • Vulnerabilities requiring physical access to the victim's unlocked device
  • Denial of Service attacks
  • Brute Force attacks
  • Spam or Social Engineering techniques, including:
  • SPF and DKIM issues
  • Content injection
  • Hyperlink injection in emails
  • IDN homograph attacks
  • RTL Ambiguity
  • Content Spoofing
  • Issues relating to Password Policy
  • Full-Path Disclosure on any property
  • Version number information disclosure
  • Clickjacking on pre-authenticated pages, or the non-existence of X-Frame-Options, or other non-exploitable clickjacking issues (An exploitable clickjacking vulnerability requires a) a frame-able page that is b) used by an authenticated user and c) which has a state-changing action on it vulnerable to clickjacking/frame re-dressing)
  • CSRF-able actions that do not require authentication (or a session) to exploit
  • Reports related to the following security-related headers: Strict Transport Security (HSTS) XSS mitigation headers (X-Content-Type and X-XSS-Protection) ** X-Content-Type-Options
  • Content Security Policy (CSP) settings (excluding nosniff in an exploitable scenario)
  • Security bugs in third-party applications or services built on INSERTENTITYNAME's APIs - please report them to the third party that built the application or service
  • Security bugs in software related to an acquisition for a period of 90 days following any public announcement
  • Former INSERTENTITYNAME employees within one year of their departure from INSERTENTITYNAME

Safe Harbor

Any activities conducted in a manner consistent with this policy will be considered authorized conduct and we will not initiate legal action against you. If legal action is initiated by a third party against you in connection with activities conducted under this policy, we will take steps to make it known that your actions were conducted in compliance with this policy.

Thank you for helping keep INSERTENTITYNAME and our users safe!

Scope

There are far too many posts about determining scope. if you are not sure what to do, the below scope and severity is a great start.

ProgramScope.PNG