- Miscellaneous
- tmux
- Tensorboard on remote server
- Mount on remote directory
- TeamViewer to a headless server
- Using SSH tunneling for SOCKS5 proxy
- Use conda-wise CUDA to complie package
Miscellaneous
Get the number of files
ls . | wc -l
ls /foo/*.imgs | wc -l
Get the disk size of folder
du -hs ./datasets
Get disk space info
df -h
Create and delete files
touch foo.txt
mkdir a_folder
rm foo.txt
rm -r a_folder
Check SSD status
check which disk current path . is placed.
df . -h
output:
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p2 3.5T 973G 2.4T 30% /
check name and rotation of all disk:
lsblk -o NAME,ROTA
output:
NAME ROTA
nvme1n1 0
└─nvme1n1p1 0
nvme0n1 0
├─nvme0n1p1 0
└─nvme0n1p2 0
ROTA=0 means the SSD, 1 denotes the HDD.
Compression
Compress and Decompress:
tar -cvzf a_folder.tar.gz a_folder
tar -xvzf a_folder.tar.gz # decompression
unzip a_folder.zip # decompression
Split and Concat:
split -b 1024m -d --verbose a_folder.tar.gz a_folder.tar.gz.part_
cat a_folder.tar.gz.part_* > a_folder.tar.gz
-b: split file by size. (use -n to split by number) -d: use numeric suffixes starting at 0, not alphabetic;
Copy huge amount of files
rsync -ahW --no-i-r --info=progress2 source destination
# push/pull to remote
rsync -ahW --no-i-r --info=progress2 source foo@158.132.21.81:/home/foo/destination
rsync -ahW --no-i-r --info=progress2 foo@158.132.21.81:/home/foo/source destination
Details
- `-a`: keep file information, including owners, permissions, etc. - ``-h``: make output human-readable. - ``-W``: copy files whole (w/o delta-xfer algorithm), faster. - ``--no-i-r``: scan files before copying, rather than at the same time. Faster when lots of files. - ``--info=progress2``: display a progress bar. - ``--dry-run``: perform a trial run that doesn’t make any changes (and produces mostly the same output as a real run). - ``source`` and ``destination``: the source file/folder and destination folder. - ``source/``: If a trailing slash added, the **content** in ``source`` will be copied into the ``destination``. So if ``destination`` doesn't exist or is empty, this works like a combination of copy and rename.Kill all python
killall python
Release GPU memory
Sometimes the GPU memory was not released after the training or a crash, which can be checked via the GPU memory assumption shown by the nvidia-smi. If you are sure that’s a python script, you may directly run killall python, which could be danguours as it will kill all python process. Or
- You can kill the process with the PID if it was shown in
nvidia-smi:kill -9 $PID - If the PID is not shown via
nvidia-smi, you may check the GPU memory via:sudo fuser -v /dev/nvidia*Run Multiple commands
You can execute multiple commands in line, e.g.:
python train1.py; python train2.py; python test.py, which will execute the commands sequentially. Available delimeters:
|: pipes (pipelines) the standard output (stdout) of one command into the standard input of another one. Note that stderr still goes into its default destination, whatever that happen to be.|&: pipes both stdout and stderr of one command into the standard input of another one. Very useful, available in bash version 4 and above.&&: executes the right-hand command of&&only if the previous one succeeded.||: executes the right-hand command of||only it the previous one failed.;: executes the right-hand command of;always regardless whether the previous command succeeded or failed. Unless set -e was previously invoked, which causes bash to fail on an error.
To run program after a exsiting command finished, one can use the short-cut Ctrl + Z to first suspend the running command and then run:
fg ; following_command
Save print output
your_command |& tee output.txt
Other syntax, copied from the stackoverflow:
|| visible in terminal || visible in file || existing
Syntax || StdOut | StdErr || StdOut | StdErr || file
==========++==========+==========++==========+==========++===========
> || no | yes || yes | no || overwrite
>> || no | yes || yes | no || append
|| | || | ||
2> || yes | no || no | yes || overwrite
2>> || yes | no || no | yes || append
|| | || | ||
&> || no | no || yes | yes || overwrite
&>> || no | no || yes | yes || append
|| | || | ||
| tee || yes | yes || yes | no || overwrite
| tee -a || yes | yes || yes | no || append
|| | || | ||
n.e. (*) || yes | yes || no | yes || overwrite
n.e. (*) || yes | yes || no | yes || append
|| | || | ||
|& tee || yes | yes || yes | yes || overwrite
|& tee -a || yes | yes || yes | yes || append
Add the current directory to PythonPath
# Windows
$env:PYTHONPATH += ";$pwd"
# Linux
PYTHONPATH=$PWD:$PYTHONPATH
Kill port
sudo lsof -i:7777
kill $PID
Print the public IP
curl ifconfig.me
tmux
start new:
tmux
start new with session name:
tmux new -s myname
attach:
tmux a (or at, or attach)
attach to named:
tmux a -t myname
list sessions:
tmux ls
kill session:
tmux kill-session -t myname
kill all sessions:
tmux kill-server
tmux command mode (inside tmux)
Ctrl + b
split current window horizontally (top-down)
ctrl + b + "
split current window vetically (left-right)
ctrl + b + %
nevigate among windows
ctrl + b + left/up/right/down
nevigate among sessions
ctrl + s + up/down
Shortcuts in tmux command mode
?: list shortcuts
s: list sessions and chose one to switch to (recommended)
d: detach
x: kill current window
c: create a window
w: list windows
new: create a new session
Tensorboard on remote server
on server:
tensorboard --logdir <path> --port 7777
on local host:
ssh -N -L 7777:localhost:7777 luchongkai@158.132.21.81
Mount on remote directory
Using the sshfs (require root). Here is an example that I want the training results produced from remote server to be saved on local machine (so I don’t have to manually copy it), referring to an external solution
localusername@localmachine: ssh username@server -R 10000:localmachine:22
username@server: cd path2remote_project/
username@server: sshfs -p 10000 -o idmap=user,nonempty localusername@127.0.0.1:path2local_project/results results
TeamViewer to a headless server
sudo apt-get install xserver-xorg-video-dummy
sudo nano /usr/share/X11/xorg.conf.d/xorg.conf
Paste the below content into the xorg.conf
Section "Device"
Identifier "DummyDevice"
Driver "dummy"
VideoRam 256000
EndSection
Section "Screen"
Identifier "DummyScreen"
Device "DummyDevice"
Monitor "DummyMonitor"
DefaultDepth 24
SubSection "Display"
Depth 24
Modes "1920x1080_60.0"
EndSubSection
EndSection
Section "Monitor"
Identifier "DummyMonitor"
HorizSync 30-70
VertRefresh 50-75
ModeLine "1920x1080" 148.50 1920 2448 2492 2640 1080 1084 1089 1125 +Hsync +Vsync
EndSection
Reboot to bingo.
Someone mentioned that instead of /usr/share/X11/xorg.conf.d/xorg.conf, create .conf at /etc/X11/xorg.conf worked for them. You may have a try.
Using SSH tunneling for SOCKS5 proxy
Refer to this answer: Can I use the SSH tunnel under a VPN as a VPN server?
Use conda-wise CUDA to complie package
We often have a cudatoolkit installed in each conda environment and a CUDA installed in the system. Their version could be different. When we complie a packages that requires CUDA, it uses the system-wise CUDA by default. Which could have version compatability problem. As change the version of system-wise CUDA is tricky and danguous. We often want to compile with the conda-wise CUDA.
You can use the below commands to add the libraries inside a conda environment to the PATH (note this works temporally and you need re-run them if you re-open another terminal):
export CUDA_HOME=/home/louis/miniconda3/envs/env_name # replace this path to your env
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib:$LD_LIBRARY_PATH
Then, run ncvv -V to check if the version is equal to the one in your environment or the system-wise one.
You may sometimes do not have a (complete) cudatoolkit inside your environment as a pytorch-installation only include neccessary cuda depencencies. In this case, you could manually install one by conda install nvidia::cuda-toolkit, refer to this site, or nvcc via conda install nvidia::cuda-nvcc, refer to this site.