The most crucial practice in cloud security is private access and a control panel for managing users with whom service users can access. However, too much control in private access and administration will cause headache for every new member when hands-on coding. Especially more services require more maintenance from both maintainer and user, and the resource for keeping everything in standard security practice requires a lot in the workforce and net cloud cost. Tailscale is the new VPN service that offers to solve this challenge at a competitive cost.
Currently, my working environment requires Bastion host and VPN to access the dev server and production database. The setup is simple at first, but as the team and service grow, the complexity of the setup is growing respectively. Hands-on new dev is never easy, and despite I already set up Dev microVM with all the necessary environment and packages, the network config is the most confusing and complicated to debug for junior dev. The current process’s problems can be listed as:
- Some HPC servers are in the office building, and team members can only access the servers via the office network. Emailing the administrator network for the working VPN is hostile, and even after requesting for years, my request has not been fulfilled cause of insufficient member requests 😥.
- I have tried Cloudflare Tunnel and Ngrok, but the setup is not instructive, and the service must be registered in the specific domain, but my only need is to access the service via email, nothing more.
- AWS cloud service of our platform is mainly in the private subnet. For the team members to access the cloud database, they must access via Bastion host or VPN.
- Bastion host will get complicated quickly as the service grows and require nested ssh config to memorize all the service IPs.
- VPN is ok, but you can only use one VPN at a time and can not access another network, such as internal HPC servers or other VPN networks.
My cram about the network configuration has been built up for years, and seeing new dev frustration when debugging local is not a pleasant sight. Tailscale is my new solution for solving all the problems I listed. The team experience has been outstanding, and everything can be accessed and configured easily.
Login tailscale is relative easy, go to this page to register. But currently taiscale only support creat account with certain SSO identity providers (eg Goole or Microsoft). If you need other support providers, please follow this instruction.
After create account and login, you will be navigate to admin page with the listing service.
The above image is the listing devices I am using; each device has its IPv4 address that can be accessed when logging into Tailscale.
Tailscale helps you connect your devices together. To regsiter your device to tailscale cluster, download tailscale clien both in your host and client device. Tailscale works seamlessly with Linux, Windows, macOS, Raspberry Pi, Android, Synology, and more. Download Tailscale and log in on the device.
Here is how I install tailscale in Fedora server:
$ curl -fsSL https://tailscale.com/install.sh | sh
$ sudo tailscale up To authenticate, visit: https://login.tailscale.com/a/abcde
Via key in your setting keys
$ sudo tailscale up --authkey tskey-abcdef1432341818
Check that you can ping your new device from your personal Tailscale machine. You can find the Tailscale IP in the admin console, or by running this command on the new register machine.
$ tailscale ip -4 100.83.201.24 # Your IPv4 in tailscale
Ping the new IPv4 register device in your client machine
$ ping 100.83.201.24 Pinging 100.83.201.24 with 32 bytes of data: Reply from 100.83.201.24: bytes=32 time=77ms TTL=255 Reply from 100.83.201.24: bytes=32 time=32ms TTL=255 Reply from 100.83.201.24: bytes=32 time=32ms TTL=255 Reply from 100.83.201.24: bytes=32 time=32ms TTL=255
If the ping process get through like the above command, you can ssh or access port in new device anywhere via IPv4 register in Tailscale Admin.
subnet router (previously called a relay node or relaynode) to access these devices from Tailscale. Subnet routers act as a gateway, relaying traffic from your Tailscale network onto your physical subnet. Subnet routers respect features like access control policies, which make it easy to migrate a large network to Tailscale without installing the app on every device.
Setting up a subnet router is relatively easy and you can read this guide for more specifics. Below is the detailed process that I set up in the AWS cloud.
First, create an EC2 instance running Amazon Linux on either x86 or ARM. Tailscale produces Linux packages containing binaries for both architectures, and the AWS ARM instances are very cost effective. When setting the security policy,
allow UDP port 41641 to ingress from any source. This will enable direct connections, to minimize latency.
Then ssh to the system and follow the steps to install Tailscale on Amazon Linux and configure subnet routing. When running
tailscale up, pass your VPC subnet address to
As the above image, the subnet range VPC is 10.0.0.0/16:
$ tailscale up --advertise-routes=10.0.0.0/16 --accept-dns=false
Check your instance router is successfully configured
$ tailscale ip -4 100.83.201.24
This following step is not required if using autoApprovers in admin page.
Open the machines page in the admin console, and locate the device that advertised subnet routes. You can look for the Subnets badge in the machines list. Using the
... icon at the end of the table, select
Edit route settings. This will open up the Edit route settings panel.
Approve each route individually by clicking the toggle to the left of the route.
You can traceroute and SSH private instances (10.0.26.12) behind the subnet route (100.83.201.24).
$ traceroute 10.0.26.12 traceroute to 10.0.26.12 (10.0.26.12), 64 hops max, 52 byte packets 1 100.83.201.24 (100.83.201.24) 27.135 ms 17.935 ms 18.342 ms 2 10.0.26.12 (10.0.26.12) 19.396 ms 18.852 ms 18.364 ms $ ssh /path/to/private_key firstname.lastname@example.org
If you need to access other services such as AWS RDS database that are created without public accessibility, the hostname is resolved with a private IP address outside the VPC. This IP address belongs to the advertised routes.
You can access RDB instances from the Tailscale network in the same way.
$ dig +short database.xxx.ap-southeast-1.rds.amazonaws.com 10.0.3.176 $ mysqlsh --email@example.com:3306 ... MySQL database-1.c75dvgjlkohb.ap-southeast-1.rds.amazonaws.com:3306 ssl JS >
tailscale up --accept-routes=trueto use the routes being advertised by the subnet router in AWS.
- Using Tailscale has been so blessed that I can not return to the traditional way anymore. Every service can be accessed by its private domain, no more mess up the environment cause the network switch or port is already in use.
- Tailscale is a paid service, and you must depend on Tailscale to manage the admin server. Thankfully, Taiscale also provides a solution to open-source hosting Headscale, and you can configure all the admin server on your own.
- For the final wrap-up, you can use Terraform to manage Tailscale and automate all the processes to setup subnet router that I listed above.