Difference between revisions of "Linux utils"
From Wasya Wiki
(→Mount a block devise) |
(→Disks, Storage) |
||
(75 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
= Utils = | = Utils = | ||
+ | * [[ css ]] | ||
+ | |||
+ | * [[ Git cheatsheet ]] | ||
+ | * [[ IO Utils ]] | ||
+ | * [[ json ]] | ||
+ | * [[ kubernetes ]] | ||
+ | * [[ Memory Utils ]] | ||
+ | |||
+ | * [[ Network Utils ]] | ||
+ | * [[ Process Utils ]] | ||
+ | * [[ xml ]] | ||
+ | |||
+ | === Replace all occurrences of a string in a folder === | ||
+ | ubuntu? | ||
+ | find /home/user/ -type f | xargs sed -i 's/a.example.com/b.example.com/g' | ||
+ | mac: | ||
+ | <pre> | ||
+ | LC_ALL=C find . -type f -name "*.html" -exec sed -i '' "s/UA-47088821-3/UA-47088821-4/" {} + | ||
+ | </pre> | ||
+ | |||
+ | === last 3 accessed files === | ||
+ | find . -type f -exec stat -c '%X %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}' | ||
+ | |||
+ | === last 3 modified files === | ||
+ | find . -type f -exec stat -c '%Y %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}' | ||
+ | |||
+ | === Find 10 largest files in a folder === | ||
+ | |||
+ | du -a /var | sort -n -r | head -n 10 | ||
+ | du -a . | sort -n -r | head -n 15 | ||
+ | |||
+ | For mac: | ||
+ | |||
+ | du -a . | sort -n -r | head | ||
+ | |||
+ | Limit how much logs journalctl keeps: | ||
+ | |||
+ | journalctl --vacuum-time=2d | ||
+ | journalctl --vacuum-size=500M | ||
+ | |||
+ | === Count Lines in a Folder === | ||
+ | find . -name "*js" -print | grep -v node_modules | xargs cat | grep -v "^//" | grep -v "^$" | wc -l | ||
+ | |||
+ | |||
+ | === cut after ag for shorter lines === | ||
+ | |||
+ | ag <your-search-keyword> | cut -c-800 | ||
+ | |||
+ | === increase number of open files, file descriptors === | ||
+ | |||
+ | From: /etc/security/limits.conf | ||
+ | <pre> | ||
+ | * - nofile 131072 | ||
+ | |||
+ | * soft nofile 64000 | ||
+ | * hard nofile 64000 | ||
+ | root soft nofile 64000 | ||
+ | root hard nofile 64000 | ||
+ | </pre> | ||
+ | |||
+ | Reload services after making changes, logout. | ||
+ | |||
+ | === count open files === | ||
+ | lsof | wc - | ||
+ | Per user: | ||
+ | lsof | grep ' opera ' | awk '{print $NF}' | sort | wc -l | ||
=== Set date === | === Set date === | ||
sudo date --set "25 Sep 2013 15:00:00" | sudo date --set "25 Sep 2013 15:00:00" | ||
− | === | + | === Get date in milliseconds === |
− | + | date -d 2017-05-05 +%s | |
− | |||
− | |||
− | + | == Disks, Storage == | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | === check | + | === check disks on ubuntu === |
− | + | ||
− | + | sudo lsblk | |
− | + | ||
− | === | + | === Mount a block device === |
− | + | ||
− | + | From: https://www.digitalocean.com/community/tutorials/how-to-use-block-storage-on-digitalocean | |
− | + | ||
− | + | parted /dev/sda mklabel gpt | |
− | + | parted -a opt /dev/sda mkpart primary ext4 0% 100% | |
+ | mkfs.ext4 /dev/sda | ||
+ | mkdir -p /data | ||
+ | '/dev/sda /data ext4 defaults,nofail,discard 0 2' | sudo tee -a /etc/fstab | ||
+ | mount -a | ||
=== Find large files === | === Find large files === | ||
Line 42: | Line 101: | ||
dd if=/dev/urandom of=/var/log/mongodb/randomFile bs=1M count=200 | dd if=/dev/urandom of=/var/log/mongodb/randomFile bs=1M count=200 | ||
− | |||
− | |||
− | |||
− | |||
− | |||
=== Create a user === | === Create a user === | ||
− | |||
su - <username> | su - <username> | ||
Line 55: | Line 108: | ||
usernameusedforlogin ALL=(ALL) NOPASSWD:ALL | usernameusedforlogin ALL=(ALL) NOPASSWD:ALL | ||
sudo adduser --disabled-password <username> | sudo adduser --disabled-password <username> | ||
+ | |||
+ | add user to group: | ||
+ | |||
+ | usermod -a -G wheel niceguy | ||
+ | useradd -g wheel niceguy ## only is user doesn't exist! | ||
+ | |||
+ | or in rhel: | ||
+ | |||
+ | passwd -f -u deploy | ||
(and there is deluser) | (and there is deluser) | ||
(I don't think I need this: usermod -aG sudo username ) - for making him a sudoer | (I don't think I need this: usermod -aG sudo username ) - for making him a sudoer | ||
− | + | useradd? adduser? | |
+ | |||
+ | adduser : add user with full profile and info (pass, quota, permission, etc.) adduser is friendlier in that it sets up the account's home folders and other settings (e.g. automatically loading system stats and notifications on login), | ||
+ | |||
+ | whereas useradd just creates the user | ||
+ | |||
+ | |||
− | + | === Delete files older than 5 days === | |
+ | find /path/to/files* -mtime +5 -exec rm {} \; | ||
+ | find /path/to/files* -type f -mtime +5 -exec rm {} -v \; | ||
− | + | === get children of a process === | |
− | + | pgrep -P $your_process1_pid | |
− | + | ||
− | === | + | === restrict resources given to a process === |
− | + | ||
− | + | ||
− | + | sudo apt-get install cgroup-bin -y | |
+ | sudo cgcreate -g memory:/rmGroup | ||
+ | sudo | ||
+ | echo $(( 500 * 1024 * 1024 )) > /sys/fs/cgroup/memory/rmGroup/memory.limit_in_bytes | ||
+ | # echo $(( 5000 * 1024 * 1024 )) > /sys/fs/cgroup/memory/rmGroup/memory.memsw.limit_in_bytes # swap only | ||
+ | # cgexec -g memory:rmGroup <command> | ||
+ | cgexec -g memory:rmGroup find cache/ -type f -mtime +10 -exec rm {} -v \; |
Latest revision as of 19:11, 29 September 2024
Contents
- 1 Utils
- 1.1 Replace all occurrences of a string in a folder
- 1.2 last 3 accessed files
- 1.3 last 3 modified files
- 1.4 Find 10 largest files in a folder
- 1.5 Count Lines in a Folder
- 1.6 cut after ag for shorter lines
- 1.7 increase number of open files, file descriptors
- 1.8 count open files
- 1.9 Set date
- 1.10 Get date in milliseconds
- 1.11 Disks, Storage
Utils
Replace all occurrences of a string in a folder
ubuntu?
find /home/user/ -type f | xargs sed -i 's/a.example.com/b.example.com/g'
mac:
LC_ALL=C find . -type f -name "*.html" -exec sed -i '' "s/UA-47088821-3/UA-47088821-4/" {} +
last 3 accessed files
find . -type f -exec stat -c '%X %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}'
last 3 modified files
find . -type f -exec stat -c '%Y %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}'
Find 10 largest files in a folder
du -a /var | sort -n -r | head -n 10 du -a . | sort -n -r | head -n 15
For mac:
du -a . | sort -n -r | head
Limit how much logs journalctl keeps:
journalctl --vacuum-time=2d journalctl --vacuum-size=500M
Count Lines in a Folder
find . -name "*js" -print | grep -v node_modules | xargs cat | grep -v "^//" | grep -v "^$" | wc -l
cut after ag for shorter lines
ag <your-search-keyword> | cut -c-800
increase number of open files, file descriptors
From: /etc/security/limits.conf
* - nofile 131072 * soft nofile 64000 * hard nofile 64000 root soft nofile 64000 root hard nofile 64000
Reload services after making changes, logout.
count open files
lsof | wc -
Per user:
lsof | grep ' opera ' | awk '{print $NF}' | sort | wc -l
Set date
sudo date --set "25 Sep 2013 15:00:00"
Get date in milliseconds
date -d 2017-05-05 +%s
Disks, Storage
check disks on ubuntu
sudo lsblk
Mount a block device
From: https://www.digitalocean.com/community/tutorials/how-to-use-block-storage-on-digitalocean
parted /dev/sda mklabel gpt parted -a opt /dev/sda mkpart primary ext4 0% 100% mkfs.ext4 /dev/sda mkdir -p /data '/dev/sda /data ext4 defaults,nofail,discard 0 2' | sudo tee -a /etc/fstab mount -a
Find large files
find / -xdev -type f -size +100M
see timestamps in history
echo 'export HISTTIMEFORMAT="%d/%m/%y %T "' >> ~/.bashrc ; source ~/.bashrc
Generate large random file
dd if=/dev/urandom of=/var/log/mongodb/randomFile bs=1M count=200
Create a user
su - <username> sudo visudo -f /etc/sudoers usernameusedforlogin ALL=(ALL) NOPASSWD:ALL sudo adduser --disabled-password <username>
add user to group:
usermod -a -G wheel niceguy useradd -g wheel niceguy ## only is user doesn't exist!
or in rhel:
passwd -f -u deploy (and there is deluser) (I don't think I need this: usermod -aG sudo username ) - for making him a sudoer
useradd? adduser?
adduser : add user with full profile and info (pass, quota, permission, etc.) adduser is friendlier in that it sets up the account's home folders and other settings (e.g. automatically loading system stats and notifications on login),
whereas useradd just creates the user
Delete files older than 5 days
find /path/to/files* -mtime +5 -exec rm {} \; find /path/to/files* -type f -mtime +5 -exec rm {} -v \;
get children of a process
pgrep -P $your_process1_pid
restrict resources given to a process
sudo apt-get install cgroup-bin -y sudo cgcreate -g memory:/rmGroup sudo echo $(( 500 * 1024 * 1024 )) > /sys/fs/cgroup/memory/rmGroup/memory.limit_in_bytes # echo $(( 5000 * 1024 * 1024 )) > /sys/fs/cgroup/memory/rmGroup/memory.memsw.limit_in_bytes # swap only # cgexec -g memory:rmGroup <command> cgexec -g memory:rmGroup find cache/ -type f -mtime +10 -exec rm {} -v \;