Implementing SafeSquid on Cloud
Background
The SafeSquid proxy server is a native x86_64 Linux network service application.
The software is distributed as a “tar-ball”
package. It can thus be deployed on an appropriate Linux based host platform.
Efficient performance installation and configuration of a few other co-resident system applications and libraries.
The SafeSquid proxy server and the other applications co-resident on the host platform can generate lots of files.
This can endanger the proxy service if the disk runs out of space.
It is therefore advisable to create appropriate disk partitions for application safety and long-term reliability.
A standard installation of SafeSquid SWG proxy server is initiated using SafeSquid’s custom ISO also known as SafeSquid Appliance Builder (SAB) which is based on ubuntu 18.04 LTS.
SAB is customized to provide automatic setup of all the necessary files and services, besides optimum disk-partitioning.
The SAB installer automatically fetches the tar-ball and deploys SafeSquid towards the end of its execution.
Installing on a Cloud PaaS such as Azure
You cannot setup SafeSquid using SAB on a PaaS such as Azure that does not allow upload of custom ISO file.
The SafeSquid tar-ball fetched by the SAB can however be manually downloaded and installed virtual guests created on such PaaS.
Some of the important functions carried out by the SAB installer must however be done manually, making this option not quite optimal.
Most importantly, implementing the optimum disk partitioning recipe can be most frustrating for a first-timer.
Function of partitioning recipe is lost in tar-ball installation.
Storing log files, user cache, database file and other SafeSquid files in shared block device as root directory, where Linux operating system and Linux kernel files reside, can lead to reduction in overall performance, cause OS failure and data corruption when disk is utilized 100%.
Users without custom SafeSquid partition witness issues where they lose log files generate by SafeSquid, Main reason for losing log files are monit’s control file for managing disk utilization.
Monit is a supervision tool which uses control file to perform automatic maintenance and such a control file is used for maintaining disk utilization, it removes old files from /var/log/safesquid and /var/db/safesquid when disk utilized is more than 80%.
Losing log data can lead to setbacks in troubleshooting process.
To avoid such issue, SafeSquid creates custom partitions using LVM, which helps isolate files in logical volumes.
Option for scalability is also lost, additional storage cannot be appended to existing block device.
Example: When using ext4 partition, disk size for / or root directory cannot be extended when OS is booted from block device.
Resizing block device requires downtime, which leads to decrement in production hours.
Example: To add additional storage space to directory /var/log/safesquid you’ll will need to power off your server, add additional storage disk, power back on the server, create filesystem, mount your block device and relocate current log files to the new partition or block device.
To overcome such drawbacks, you can use cloud-init to deploy SafeSquid instance with custom partitions in Microsoft Azure.
Below are steps following which you can install SafeSquid via cloud-init
Deploying SafeSquid in Microsoft Azure.
Access your Microsoft Azure dashboard at https://portal.azure.com/#home
SafeSquid's Cloud-init
Personaliztion
Fqdn, hostname timezone and keyboard layout should be updated as per your proxy environment.
Custom Logical Volume
Default values for the logical volume is set to minimum disk space required for each partition.
Note: the given values for the disk sizes are in megabytes.
Update the (--size, “512”) part of the command with the required disk size as per your requirements.
For example: Update the size of /var/lib/safesquid from 1GB to 3 GB
Update
- [ lvcreate, --size, '1024', --name, /dev/azure-vg/var+lib+safesquid, azure-vg ]
to
- [ lvcreate, --size, '3072', --name, /dev/azure-vg/var+lib+safesquid, azure-vg ]
Minimum disk space required to create disk partition is a minimum of 16GB.
When using disk with storage capacity more the 250GB, refer to the recommended maximum disk space.
Recommended maximum disk space required for SafeSquid partition are
4096 - /opt/safesquid
4096 - /usr/local/safesquid
8192 - /var/lib/safesquid
8192 - /var/www/safesquid
Allotting space more than recommended maximum disk space for mentioned partitions may not be optimal.
Directories /var/db/safesquid, /var/cache/safesquid and /var/log/safesquid, does not have maximum disk size.
You can set maximum disk size for each partition as required
NOTE: Sections with ##Do not modify, should not be updated, unless you are aware of the components you are modifying.
#cloud-config #Set HostName fqdn: azure.local.domain hostname: azure prefer_fqdn_over_hostname: true #Manage etc hosts file, add entry in hostfile ${fqdn} 127.0.0.1 ##Do not modify. manage_etc_hosts: localhost #Set TimeZone and Keyboard layout timezone: Asia/Kolkata keyboard: layout: us #Create User administrator and ssquid. users: #Default user Created during the deployment process, ##Do not modify. - default #Default SafeSquid admin user, ##Do not modify. - name: administrator gecos: administrator groups: admin sudo: ALL=(ALL) ALL shell: /bin/bash #User for SafeSquid Process, ##Do not modify. - name: ssquid gecos: ssquid groups: root sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/sh #expire password for administrator after first login, ##Do not modify. chpasswd: expire: true users: - name: administrator password: safesquid runcmd: #Create Custom LVM Partitions # [H:C:T:L] ## 'H' = host number## 'C' = controller number## 'T' = target number## 'L'= Logical Unit Number (LUN) (3-255) #Create Physical Volume - lsscsi '[*:*:*:3]' | awk '{print $7}' | xargs pvcreate #Create Volume Group - lsscsi '[*:*:*:3]' | awk '{print $7}' | xargs vgcreate azure-vg #Create Logical Volume - [ lvcreate, --size, '512', --name, /dev/azure-vg/opt+safesquid, azure-vg ] - [ lvcreate, --size, '512', --name, /dev/azure-vg/var+www+safesquid, azure-vg ] - [ lvcreate, --size, '1024', --name, /dev/azure-vg/usr+local+safesquid, azure-vg ] - [ lvcreate, --size, '1024', --name, /dev/azure-vg/var+lib+safesquid, azure-vg ] - [ lvcreate, --size, '1024', --name, /dev/azure-vg/var+cache+safesquid, azure-vg ] - [ lvcreate, --size, '2048', --name, /dev/azure-vg/var+db+safesquid, azure-vg ] - [ lvcreate, --size, '5120', --name, /dev/azure-vg/var+log+safesquid, azure-vg ] #Make File system, ##Do not modify. - [ mkfs.ext4, /dev/azure-vg/var+log+safesquid ] - [ mkfs.ext4, /dev/azure-vg/var+db+safesquid ] - [ mkfs.ext4, /dev/azure-vg/var+cache+safesquid ] - [ mkfs.ext4, /dev/azure-vg/var+lib+safesquid ] - [ mkfs.ext4, /dev/azure-vg/var+www+safesquid ] - [ mkfs.ext4, /dev/azure-vg/usr+local+safesquid ] - [ mkfs.ext4, /dev/azure-vg/opt+safesquid ] #Create Dir if not found, ##Do not modify. - [ mkdir, --parents, /var/log/safesquid ] - [ mkdir, --parents, /var/db/safesquid ] - [ mkdir, --parents, /var/lib/safesquid ] - [ mkdir, --parents, /var/cache/safesquid ] - [ mkdir, --parents, /var/www/safesquid ] - [ mkdir, --parents, /usr/local/safesquid ] - [ mkdir, --parents, /opt/safesquid ] #Mount Partitions, ##Do not modify. - [ mount, /dev/azure-vg/var+log+safesquid, /var/log/safesquid ] - [ mount, /dev/azure-vg/var+db+safesquid, /var/db/safesquid ] - [ mount, /dev/azure-vg/var+cache+safesquid, /var/cache/safesquid ] - [ mount, /dev/azure-vg/var+lib+safesquid, /var/lib/safesquid ] - [ mount, /dev/azure-vg/var+www+safesquid, /var/www/safesquid ] - [ mount, /dev/azure-vg/usr+local+safesquid, /usr/local/safesquid ] - [ mount, /dev/azure-vg/opt+safesquid, /opt/safesquid ] #Install SafeSquid, ##Do not modify. - [ wget, "https://downloads.safesquid.com/appliance/binary/safesquid_latest.tar.gz", -O, /tmp/safesquid_latest.tar.gz ] - [ tar, -xzvf, /tmp/safesquid_latest.tar.gz, -C, /tmp/ ] - [ bin/bash, /tmp/_mkappliance/installation/setup.sh ] #Get msktutil required for kerberos, ##Do not modify. - [ wget, "https://downloads.safesquid.com/appliance/source/msktutil", -O, /usr/local/bin/msktutil] - [ chmod, "0755", /usr/local/bin/msktutil] #Setup MOTD, ##Do not modify. - | echo " \033[1;32m _____ __ _____ _ _ / ____| / _| / ____| (_) | | | (___ __ _ | |_ ___ | (___ __ _ _ _ _ __| | \___ \ / _\ | | _| / _ \ \___ \ / _\ | | | | | | | / _\ | ____) | | (_| | | | | __/ ____) | | (_| | | |_| | | | | (_| | |_____/ \ _,_| |_| \___| |_____/ \__, | \__,_| |_| \__,_| | | |_| _____ __ __ _____ / ____| \ \ / / / ____| | (___ \ \ /\ / / | | ___ \___ \ \ \/ \/ / | | |_ | ____) | \ /\ / | |___| | |_____/ \/ \/ \_____/ Built on `date "+%d %B %Y"` \033[0;35m+++++++++++++: \033[0;37mHelpful Information\033[0;35m :+++++++++++++++ \033[0;35m+ \033[0;37mWeb \033[0;35m# \033[1;32mhttps://www.safesquid.com/ \033[0;35m+ \033[0;37mEMail \033[0;35m# \033[1;32msupport@safesquid.net \033[0;35m+ \033[0;37mskype \033[0;35m# \033[1;32mSafeSquid \033[0;35m+++++++++++++++++: \033[0;37mSystem Data\033[0;35m :+++++++++++++++++++ + \033[0;37mFqdn \033[0;35m= \033[1;32m`hostname -f` \033[0;35m+ \033[0;37mAddress \033[0;35m= \033[1;32m`hostname -I` \033[0;35m+ \033[0;37mKernel \033[0;35m= \033[1;32m`uname -r` \033[0;35m+ \033[0;37mMemory \033[0;35m= \033[1;32m`cat /proc/meminfo | grep MemTotal | awk {'print $2'}` kB \033[0;35m+++++++++++: \033[0;31mMaintenance Information\033[0;35m :+++++++++++++ +\033[0;31m `cat /etc/motd-maintenance` \033[0;35m+++++++++++++++++++++++++++++++++++++++++++++++++++\033[0;37m " > /etc/motd #Enable/Start/Restart services, ##Do not modify. - [ systemctl, start, bind9-resolvconf ] - [ systemctl, enable, bind9-resolvconf ] - [ systemctl, restart, bind9.service ] - [ systemctl, restart, monit.service ] #Installing debian packages, ##Do not remove any packages. packages: - aptitude - debconf - debconf-utils - dpkg - update-motd - perl-base - plymouth-themes - zlib1g - tar - heimdal-clients - libsasl2-modules-gssapi-heimdal - libgssapi3-heimdal - libkrb5-26-heimdal - libsasl2-modules-ldap - libudns0 - libpam0g - libcap-ng0 - libcap2-bin - libmagic1 - ntp - ntpdate - curl - vim - wget - gnuplot-nox - bind9 - bind9utils - bind9-host - resolvconf - ifupdown - monit - sqlite3 - libkeepalive0 - clamav-daemon - openssh-server - tree #Creating & edit Files #Fstab Entry, ##Do not modify. write_files: - content: | /dev/azure-vg/var+log+safesquid /var/log/safesquid ext4 defaults 0 0 /dev/azure-vg/var+db+safesquid /var/db/safesquid ext4 defaults 0 0 /dev/azure-vg/var+cache+safesquid /var/cache/safesquid ext4 defaults 0 0 /dev/azure-vg/var+lib+safesquid /var/lib/safesquid ext4 defaults 0 0 /dev/azure-vg/var+www+safesquid /var/www/safesquid ext4 defaults 0 0 /dev/azure-vg/usr+local+safesquid /usr/local/safesquid ext4 defaults 0 0 /dev/azure-vg/opt+safesquid /opt/safesquid ext4 defaults 0 0 path: /etc/fstab permissions: '0644' append: true #Set Limits, ##Do not modify. - content: | ulimit -HSn 8192 path: /root/.bashrc owner: root:root permissions: '0644' append: true - content: | * hard nofile 8192 path: /etc/security/limits.conf owner: root:root permissions: '0644' append: true - content: | * soft nofile 8192 path: /etc/security/limits.conf owner: root:root permissions: '0644' append: true #Create an empty file, ##Do not modify. - content: | path: /etc/motd-maintenance owner: root:root permissions: '0644' - content: | path: /etc/bind/safesquid.dns.conf owner: root:root permissions: '0644' #Set Issue, ##Do not modify. - content: | Thank You! for choosing SafeSquid Appliance This Appliance has been built using %v Username : administrator Password : safesquid You need to reset password on first login path: /etc/issue owner: root:root permissions: '0644' - content: | Thank You! for choosing SafeSquid Appliance This Appliance has been built using %v Username : administrator Password : safesquid You need to reset password on first login path: /etc/issue.net owner: root:root permissions: '0644' #Edit Bind9, ##Do not modify. - content: | options { directory "/var/cache/bind"; // If there is a firewall between you and nameservers you want // to talk to, you may need to fix the firewall to allow multiple // ports to talk. See http://www.kb.cert.org/vuls/id/800113 // If your ISP provided one or more IP addresses for stable // nameservers, you probably want to use them as forwarders. // Uncomment the following block, and insert the addresses replacing // the all-0's placeholder. // forwarders { // 0.0.0.0; // }; //======================================================================== // If BIND logs error messages about the root key being expired, // you will need to update your keys. See https://www.isc.org/bind-keys //======================================================================== //dnssec-validation auto; dnssec-validation no; max-cache-ttl 300; max-ncache-ttl 300; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; }; path: /etc/bind/named.conf.options owner: root:root permissions: '0644' - content: | include "/etc/bind/safesquid.dns.conf"; path: /etc/bind/named.conf owner: root:root permissions: '0644' append: true defer: true #Edit resolvconf, ##Do not modify. - content: | TRUNCATE_NAMESERVER_LIST_AFTER_LOOPBACK_ADDRESS=yes path: /etc/default/resolvconf append: true defer: true #Make First Boot Script, ##Do not modify. - content: | #!/bin/bash RESET_ISSUE() { /usr/bin/head -n 3 /etc/issue > /etc/issue.net cp /etc/issue.net /etc/issue } MAIN() { RESET_ISSUE } MAIN mv /usr/local/src/first_boot.sh /usr/local/src/_first_boot.sh path: /usr/local/src/first_boot.sh owner: root:root permissions: '0755' #Set First Boot, ##Do not modify. - content: | [ -f "/usr/local/src/first_boot.sh" ] && /usr/local/src/first_boot.sh path: /root/.bashrc append: true #Update && Upgrade packages, ##Do not modify. package_update: true package_upgrade: true