Linux utils

Utils

• Git cheatsheet
• Memory Utils
• utils-network
• Process Utils
• IO Utils

Replace all occurrences of a string in a folder

ubuntu?

find /home/user/ -type f | xargs sed -i  's/a.example.com/b.example.com/g'

mac:

 LC_ALL=C find . -type f -name "*.html" -exec sed -i '' "s/UA-47088821-3/UA-47088821-4/" {} +

Find 10 largest files in a folder

du -a /var | sort -n -r | head -n 10

Count Lines in a Folder

find . -name "*js" -print | grep -v node_modules | xargs cat | grep -v "^//" | grep -v "^$" | wc -l

System

Set date

sudo date --set "25 Sep 2013 15:00:00"

Get date in milliseconds

date -d 2017-05-05 +%s

Disks, Storage

check disks on ubuntu

sudo lsblk

Mount a block devise

From: https://www.digitalocean.com/community/tutorials/how-to-use-block-storage-on-digitalocean

parted /dev/sda mklabel gpt
parted -a opt /dev/sda mkpart primary ext4 0% 100% 
mkfs.ext4 /dev/sda
mkdir -p /data
'/dev/sda /data ext4 defaults,nofail,discard 0 2' | sudo tee -a /etc/fstab
mount -a

Network

check that a node is pingable in bash

while ! ping -c1 bjjc_spec_22.ubuntu14 &>/dev/null; do :; done ;

how to tell if port 80 is listening?

telnet?

how do I know if the system trusts the certificate?

you use curl.

how to tell if port 80 is listening? What is listening on a particular port?

netstat -plnt

or

lsof -nP -i4TCP:$PORT | grep LISTEN

ubuntu vpn

https://hide.me/en/vpnsetup/ubuntu/openvpn/

Find large files

find / -xdev -type f -size +100M

see timestamps in history

echo 'export HISTTIMEFORMAT="%d/%m/%y %T "' >> ~/.bashrc ; source ~/.bashrc

Generate large random file

dd if=/dev/urandom of=/var/log/mongodb/randomFile bs=1M count=200

Add ssh key to a server

cat ~/.ssh/id_rsa.pub | ssh user@hostname 'cat >> .ssh/authorized_keys'

Add identity to a server

ssh-copy-id -i ~.ssh/key_name.pub user@host

Create a user

useradd? adduser?

su - <username>
sudo visudo -f /etc/sudoers
usernameusedforlogin ALL=(ALL) NOPASSWD:ALL
sudo adduser --disabled-password <username>
(and there is deluser)
(I don't think I need this: usermod -aG sudo username ) - for making him a sudoer

Establish a simple ssh port forward

10.0.0.x is public, 10.0.1.x is private. Do:

sudo sysctl net.ipv4.ip_forward=1
sudo iptables -t nat -A PREROUTING -p tcp -d 10.0.0.229 --dport 2270 -j DNAT --to-destination 10.0.1.224:22
sudo iptables -t nat -A POSTROUTING -j MASQUERADE

Reset Chef UI Password

sudo -u opscode-pgsql  /opt/chef-server/embedded/bin/psql opscode_chef
update osc_users set hashed_password = '$2a$12$y31Wno2MKiGXS3FSgVg5UunKG48gJz0pRV//RMy1osDxVbrb0On4W' , salt ='$2a$12$y31Wno2MKiGXS3FSgVg5Uu' where username ='admin';

and login with user 'admin' and password 'password'.

mongo add member to cluster

operaeventrsX:PRIMARY> rs.add("10.138.96.89")
operaeventrsX:PRIMARY> rs.conf()

install redis

From: https://www.digitalocean.com/community/tutorials/how-to-configure-a-redis-cluster-on-ubuntu-14-04

sudo add-apt-repository ppa:chris-lea/redis-server
sudo apt-get update
sudo apt-get install redis-server
redis-benchmark -q -n 1000 -c 10 -P 5

Delete files older than 5 days

find /path/to/files* -mtime +5 -exec rm {} \;
find /path/to/files* -type f -mtime +5 -exec rm {} -v \;

get children of a process

pgrep -P $your_process1_pid

restrict resources given to a process

sudo apt-get install cgroup-bin -y
sudo cgcreate -g memory:/rmGroup
sudo
 echo $(( 500 * 1024 * 1024 )) > /sys/fs/cgroup/memory/rmGroup/memory.limit_in_bytes
 # echo $(( 5000 * 1024 * 1024 )) > /sys/fs/cgroup/memory/rmGroup/memory.memsw.limit_in_bytes # swap only
# cgexec -g memory:rmGroup <command>
cgexec -g memory:rmGroup find cache/ -type f -mtime +10 -exec rm {} -v \;

System

increase number of open files, file descriptors

From: /etc/security/limits.conf

* - nofile 131072

*    soft nofile 64000
*    hard nofile 64000
root soft nofile 64000
root hard nofile 64000

Reload services after making changes, logout.

count open files

lsof | wc -

Per user:

lsof | grep ' opera ' | awk '{print $NF}' | sort | wc -l