1.1.1 Essential duties of the system administrator

python -c "import spacy; spacy.load('en'); print('OK')"

shutdown -r +5 "Server will restart in 5 minutes. Please save your work."

sudo apt-get install openssh-server

sudo apt-get install subversion
svn checkout http://192.168.1.30/svn/gftchina/Hanadev/trunk/WaveFront/

1.1.2 Suggested background

1.1.3 Friction between UNIX and Linux

1.1.4 Linux distributions

cat /etc/redhat-release
uname -m && cat /etc/*release
lsb_release -a

1.1.5 Example systems used in this book

1.1.6 System-specific administration tools

1.1.7 Notation and typographical conventions

1.1.8 Units

1.1.9 Man pages and other on-line documentation

1.1.10 Other authoritative documentation

1.1.11 Other sources of information

1.1.12 Ways to find and install software

1.1.13 System administration under duress

1.1.14 Configuring a deep learning machine after installing

1. install cuda & cudnn on cent os
   
   1. yum remove "nvidia"

   sh cuda_9.0.176_384.81_linux.run

   1. nvcc–version
   2. yum install ./libcudnn7-7.6.3.30-1.cuda9.0.x86_64.rpm
   3. rpm -ivh libcudnn7-*.x86_64.rpm
   4. rpm -ivh libcudnn7-devel-*.x86_64.rpm
   5. rpm -ivh libcudnn7-doc-*.x86_64.rpm
   6. cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
   7. tar -xzvf cudnn-9.0-linux-x64-v7.1.tgz
   8. cp cuda/include/cudnn.h /usr/local/cuda/include
   9. cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
   10. chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*
2. use conda
   
   1. remove cuda
   2. downgrade nvidia to 384.69
   3. conda create -n tfgpu python=3.6.8
   4. conda activate tfgpu
   5. conda install cuda=9.0
   6. conda install tensorflow-gpu ipython pytorch
3. a new operation system:
   

   very important: Can't use the recommended graphical driver from the system.

   1. change software update sauce to Chinese local sauce.

   sudo apt-get update
   # sudo apt-get upgrade
   

   1. install required libraries

   create example.list:

   touch example.list
   

   g++ freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-dev
   build-essential cmake git unzip pkg-config python-pip openssh-server
   libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
   libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
   libxvidcore-dev libx264-dev
   libgtk-3-dev
   libhdf5-serial-dev graphviz
   libopenblas-dev libatlas-base-dev gfortran
   python2.7-dev pythoexport PATH=/usr/local/cuda/bin:$PATH
   export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATHn3-dev
   linux-image-generic linux-image-extra-virtual
   linux-source linux-headers-generic
   libglu1-mesa libxi-dev libxmu-dev libglu1-mesa-dev
   freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx   (setenv "WORKON_HOME" "~/anaconda3/envs")
   libglu1-mesa libglu1-mesa-dev libglfw3-dev libgles2-mesa-dev
   

   run this line to install above libraries:

   cat example.list | xargs sudo apt-get -y install
   
   

   1. additional software:

   # sudo apt-get install python-tk python3-tk python-imaging-tk
   sh -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"
   sudo apt-get install dkms build-essential linux-headers-generic
   sudo apt-get install python-setuptools python-dev build-essential
   sudo easy_install pip
   sudo pip install --upgrade virtualenv
   
   

   1. enable remote control

   sudo apt-get install vinagre
   

   1. install nvidia driver

   check nvidia driver:

   cat /proc/driver/nvidia/version
   lspci | grep -i nvidia
   lsmod | grep nvidia
   

4. Setup Linux Nvidia GPU
   
   1. check & install linux headers

   # list linux kernal header version
   ls -l /usr/src/kernels/
   uname -r # which should equal to previous result
   # centos
   yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
   # ubuntu
   sudo apt-get install linux-headers-$(uname -r)
   

   1. Perform the following steps to install CUDA and verify the installation.

   2.1 uninstall nvidia driver first:

   To remove CUDA Toolkit: $ sudo yum remove "cublas" "cuda*" To remove NVIDIA Drivers: $ sudo yum remove "nvidia"

   1. download cuda

   cuda_downloads

   tutorial of installing cuda gpu for deep learning with python

   https://wiki.zthxxx.me/wiki/%E6%8A%80%E6%9C%AF%E5%BC%80%E5%8F%91/Linux/Ubuntu/Ubuntu-16-%E5%BC%80%E6%9C%BA%E9%BB%98%E8%AE%A4%E5%91%BD%E4%BB%A4%E8%A1%8C%E7%95%8C%E9%9D%A2/

   1. Disable the Nouveau drivers:

   Create a file at /etc/modprobe.d/blacklist-nouveau.conf with the following contents:

   blacklist nouveau
   options nouveau modeset=0
   
   

   1. Regenerate the kernel initramfs:

   sudo update-initramfs -u
   
   

   1. Reboot into runlevel 3 by temporarily adding the number "3" and the word "nomodeset" to the end of the system's kernel boot parameters.

   sudo systemctl isolate multi-user.target
   sudo systemctl enable multi-user.target
   sudo systemctl set-default multi-user.target
   sudo reboot
   

   1. install nvidia graph card driver first.

   cuda 9 need nvidia driver above 384.00.

   1. Run the installer to install with the default selections (implies acceptance of the EULA):

   sudo init 3
   # carefully choosing options when you have multiple cards installed.
   sudo sh cuda_9.1.85_387.26_linux.run
   
   

   ***WARNING: Incomplete installation! This installation did not install the CUDA Driver.
   A driver of version at least 384.00 is required for CUDA 9.1 functionality to work.
   

   1. Create an xorg.conf file to use the NVIDIA GPU for display:

   sudo nvidia-xconfig
   

   1. Reboot the system to load the graphical interface.

   Set up the development environment by modifying the PATH and LD_LIBRARY_PATH variables:

   # Please make sure that
   # -   PATH includes /usr/local/cuda-9.1/bin
   # -   LD_LIBRARY_PATH includes /usr/local/cuda-9.1/lib64, or,
   # add /usr/local/cuda-9.1/lib64 to /etc/ld.so.conf and run ldconfig as root
   export PATH=/usr/local/cuda-9.1/bin${PATH:+:${PATH}}
   export LD_LIBRARY_PATH=/usr/local/cuda-9.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
   LD_LIBRARY_PATH="/usr/local/cuda/lib64"; export LD_LIBRARY_PATH
   PATH="$PATH:/usr/local/cuda/bin";export PATH
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
   source ~/.bashrc
   exec "$SHELL"
   

   • Install a writable copy of the samples then build and run the nbody sample:

   cuda-install-samples-9.1.sh ~
   cd ~/NVIDIA_CUDA-9.1_Samples/5_Simulations/nbody
   GLPATH=/usr/lib make
   ./nbody
   

   1. in case you need to kill x service manually:

   sudo systemctl isolate graphical.target sudo systemctl enable graphical.target sudo systemctl set-default graphical.target sudo reboot

   1. remove nvidia

   To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-9.1/bin
   To uninstall the NVIDIA Driver, run nvidia-uninstall
   
   Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-9.1/doc/pdf for detailed information on setting up CUDA.
   sudo /usr/local/cuda-X.Y/bin/uninstall_cuda_X.Y.pl
   sudo apt-get remove --purge nvidia*
   sudo apt-get update
   sudo apt-get install dkms build-essential linux-headers-generic
   

   1. cuda version

   nvcc --version
   

   1. cudnn version

   cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
   

   1. install cudnn:
      
      1. mount share folder:

      sudo mount -t cifs -o username="gft" //192.168.1.126/share ~/Downloads/share
      

      1. unzip the cuDNN package

      tar -xzvf cudnn-9.1-linux-x64-v7.tgz
      

      1. Copy the following files into the CUDA Toolkit directory.

      sudo cp cuda/include/cudnn.h /usr/local/cuda/include
      sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
      sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*
      

      1. install CUDA Toolkit

      # best way
      sudo apt install nvidia-cuda-toolkit
      # or
      wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.1.85-1_amd64.deb
      sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb
      sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
      sudo apt-get update
      sudo apt-get install cuda
      

      1. check nvidia driver installation

      nvidia-smi
      

      1. install cuDNN source code:

      oem@dl ~/Downloads $ sudo dpkg -i libcudnn7_7.0.5.15-1+cuda9.1_amd64.deb
      

      Selecting previously unselected package libcudnn7.
      (Reading database ... 265428 files and directories currently installed.)
      Preparing to unpack libcudnn7_7.0.5.15-1+cuda9.1_amd64.deb ...
      Unpacking libcudnn7 (7.0.5.15-1+cuda9.1) ...
      Setting up libcudnn7 (7.0.5.15-1+cuda9.1) ...
      Processing triggers for libc-bin (2.23-0ubuntu3) ...
      oem@dl ~/Downloads $ sudo dpkg -i libcudnn7-dev_7.0.5.15-1+cuda9.1_amd64.deb
      Selecting previously unselected package libcudnn7-dev.
      (Reading database ... 265435 files and directories currently installed.)
      Preparing to unpack libcudnn7-dev_7.0.5.15-1+cuda9.1_amd64.deb ...
      Unpacking libcudnn7-dev (7.0.5.15-1+cuda9.1) ...
      Setting up libcudnn7-dev (7.0.5.15-1+cuda9.1) ...
      update-alternatives: using /usr/include/x86_64-linux-gnu/cudnn_v7.h to provide /usr/include/cudnn.h (libcudnn) in auto mode
      oem@dl ~/Downloads $ sudo dpkg -i libcudnn7-doc_7.0.5.15-1+cuda9.1_amd64.deb
      (Reading database ... 265441 files and directories currently installed.)
      Preparing to unpack libcudnn7-doc_7.0.5.15-1+cuda9.1_amd64.deb ...
      Unpacking libcudnn7-doc (7.0.5.15-1+cuda9.1) over (7.0.5.15-1+cuda9.1) ...
      Setting up libcudnn7-doc (7.0.5.15-1+cuda9.1) ...
      oem@dl ~/Downloads $ ls
      cuda                          install.sh                                  libcudnn7-doc_7.0.5.15-1+cuda9.1_amd64.deb
      cuda_9.1.85_387.26_linux.run  libcudnn7_7.0.5.15-1+cuda9.1_amd64.deb      NVIDIA-Linux-x86_64-384.98.run
      cudnn-9.1-linux-x64-v7.tgz    libcudnn7-dev_7.0.5.15-1+cuda9.1_amd64.deb  share
      oem@dl ~/Downloads $ cp -r /usr/src/cudnn_samples_v7/ /home/oem/projects/
      oem@dl ~/Downloads $ cd ../projects/
      oem@dl ~/projects $ ls
      cudnn_samples_v7
      oem@dl ~/projects $ cd cudnn_samples_v7/
      oem@dl ~/projects/cudnn_samples_v7 $ ls
      conv_sample  mnistCUDNN  RNN
      oem@dl ~/projects/cudnn_samples_v7 $ cd mnistCUDNN/
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ make clean &&make
      rm -rf *o
      rm -rf mnistCUDNN
      /usr/local/cuda/bin/nvcc -ccbin g++ -I/usr/local/cuda/include -IFreeImage/include  -m64    -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_53,code=sm_53 -gencode arch=compute_53,code=compute_53 -o fp16_dev.o -c fp16_dev.cu
      g++ -I/usr/local/cuda/include -IFreeImage/include   -o fp16_emu.o -c fp16_emu.cpp
      g++ -I/usr/local/cuda/include -IFreeImage/include   -o mnistCUDNN.o -c mnistCUDNN.cpp
      /usr/local/cuda/bin/nvcc -ccbin g++   -m64      -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_53,code=sm_53 -gencode arch=compute_53,code=compute_53 -o mnistCUDNN fp16_dev.o fp16_emu.o mnistCUDNN.o  -LFreeImage/lib/linux/x86_64 -LFreeImage/lib/linux -lcudart -lcublas -lcudnn -lfreeimage -lstdc++ -lm
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ ls
      data          fp16_dev.cu  fp16_dev.o    fp16_emu.h  FreeImage  Makefile    mnistCUDNN.cpp  readme.txt
      error_util.h  fp16_dev.h   fp16_emu.cpp  fp16_emu.o  gemv.h     mnistCUDNN  mnistCUDNN.o
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ ./mnistCUDNN
      ./mnistCUDNN: error while loading shared libraries: libcudart.so.9.1: cannot open shared object file: No such file or directory
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ ls /usr/local/cuda/lib64 | grep libcudart
      libcudart.so
      libcudart.so.9.1
      libcudart.so.9.1.85
      libcudart_static.a
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ PATH="$PATH:/usr/local/cuda/bin";export PATH
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ LD_LIBRARY_PATH="/usr/local/cuda/lib64"; export LD_LIBRARY_PATH
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ exec "$SHELL"
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ LD_LIBRARY_PATH="/usr/local/cuda/lib64"
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ echo $LD_LIBRARY_PATH
      /usr/local/cuda/lib64
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
      oem@dl ~/projects/cudnn_samples_v7/mnistCUDNN $ ./mnistCUDNN
      cudnnGetVersion() : 7005 , CUDNN_VERSION from cudnn.h : 7005 (7.0.5)
      Host compiler version : GCC 5.4.0
      There are 2 CUDA capable devices on your machine :
      device 0 : sms 28  Capabilities 6.1, SmClock 1582.0 Mhz, MemSize (Mb) 11172, MemClock 5505.0 Mhz, Ecc=0, boardGroupID=0
      device 1 : sms 28  Capabilities 6.1, SmClock 1582.0 Mhz, MemSize (Mb) 11169, MemClock 5505.0 Mhz, Ecc=0, boardGroupID=1
      Using device 0
      
      Testing single precision
      Loading image data/one_28x28.pgm
      Performing forward propagation ...
      Testing cudnnGetConvolutionForwardAlgorithm ...
      Fastest algorithm is Algo 1
      Testing cudnnFindConvolutionForwardAlgorithm ...
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 0: 0.031744 time requiring 0 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 1: 0.038912 time requiring 3464 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 2: 0.056320 time requiring 57600 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 7: 0.083968 time requiring 2057744 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 5: 0.108416 time requiring 203008 memory
      Resulting weights from Softmax:
      0.0000000 0.9999399 0.0000000 0.0000000 0.0000561 0.0000000 0.0000012 0.0000017 0.0000010 0.0000000
      Loading image data/three_28x28.pgm
      Performing forward propagation ...
      Resulting weights from Softmax:
      0.0000000 0.0000000 0.0000000 0.9999288 0.0000000 0.0000711 0.0000000 0.0000000 0.0000000 0.0000000
      Loading image data/five_28x28.pgm
      Performing forward propagation ...
      Resulting weights from Softmax:
      0.0000000 0.0000008 0.0000000 0.0000002 0.0000000 0.9999820 0.0000154 0.0000000 0.0000012 0.0000006
      
      Result of classification: 1 3 5
      
      Test passed!
      
      Testing half precision (math in single precision)
      Loading image data/one_28x28.pgm
      Performing forward propagation ...
      Testing cudnnGetConvolutionForwardAlgorithm ...
      Fastest algorithm is Algo 1
      Testing cudnnFindConvolutionForwardAlgorithm ...
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 0: 0.028512 time requiring 0 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 1: 0.031744 time requiring 3464 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 2: 0.052224 time requiring 28800 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 7: 0.074752 time requiring 2057744 memory
      ^^^^ CUDNN_STATUS_SUCCESS for Algo 5: 0.106496 time requiring 203008 memory
      Resulting weights from Softmax:
      0.0000001 1.0000000 0.0000001 0.0000000 0.0000563 0.0000001 0.0000012 0.0000017 0.0000010 0.0000001
      Loading image data/three_28x28.pgm
      Performing forward propagation ...
      Resulting weights from Softmax:
      0.0000000 0.0000000 0.0000000 1.0000000 0.0000000 0.0000714 0.0000000 0.0000000 0.0000000 0.0000000
      Loading image data/five_28x28.pgm
      Performing forward propagation ...
      Resulting weights from Softmax:
      0.0000000 0.0000008 0.0000000 0.0000002 0.0000000 1.0000000 0.0000154 0.0000000 0.0000012 0.0000006
      
      Result of classification: 1 3 5
      
      Test passed!
      

      1. install tensorflow with GPU support

      sudo apt-get install libcupti-dev
      

      tensorflow-gpu is not supported installed from pip, you should try building wheel from the source code from https://pypi.python.org/pypi/tensorflow-gpu. https://github.com/mind/wheels/releases/tag/tf1.4.1-gpu-cuda91

      These wheels contain MKL support. If you don't have it, install MKL by following the instructions here.

      pip --no-cache-dir install https://github.com/mind/wheels/releases/download/tf1.4.1-gpu-cuda91/tensorflow-1.4.1-cp27-cp27mu-linux_x86_64.whl --upgrade
      Collecting tensorflow==1.4.1 from https://github.com/mind/wheels/releases/download/tf1.4.1-gpu-cuda91/tensorflow-1.4.1-cp27-cp27mu-linux_x86_64.whl
      pip install --upgrade https://pypi.python.org/packages/55/68/d99d2a93e5ea22eb410e5e173d014d926ce30aa10a3d2ed7e9033deeb563/tensorflow_gpu-1.4.1-cp35-cp35m-manylinux1_x86_64.whl#md5=1cda67643cfca3feef0fea9968864bf3
      

      • verifying tensorflow-gpu library:

      python -m tensorflow.models.image.mnist.convolutional
      

      or

      git clone https://github.com/tensorflow/models.git
      workon dl4_w2v_p3
      python models/tutorials/image/mnist/convolutional.py
      

      Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
      Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
      Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
      Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
      Extracting data/train-images-idx3-ubyte.gz
      Extracting data/train-labels-idx1-ubyte.gz
      Extracting data/t10k-images-idx3-ubyte.gz
      Extracting data/t10k-labels-idx1-ubyte.gz
      2017-12-27 09:49:37.782681: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
      name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
      pciBusID: 0000:03:00.0
      totalMemory: 10.91GiB freeMemory: 10.75GiB
      2017-12-27 09:49:38.016990: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 1 with properties:
      name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
      pciBusID: 0000:04:00.0
      totalMemory: 10.91GiB freeMemory: 10.65GiB
      2017-12-27 09:49:38.017581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Device peer to peer matrix
      2017-12-27 09:49:38.017607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1051] DMA: 0 1
      2017-12-27 09:49:38.017613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 0:   Y Y
      2017-12-27 09:49:38.017617: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1061] 1:   Y Y
      2017-12-27 09:49:38.017629: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:03:00.0, compute capability: 6.1)
      2017-12-27 09:49:38.017650: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:04:00.0, compute capability: 6.1)
      Initialized!
      Step 0 (epoch 0.00), 26.8 ms
      Minibatch loss: 8.334, learning rate: 0.010000
      Minibatch error: 85.9%
      Validation error: 84.6%
      Step 100 (epoch 0.12), 4.6 ms
      Minibatch loss: 3.241, learning rate: 0.010000
      Minibatch error: 7.8%
      Validation error: 7.7%
      Step 200 (epoch 0.23), 4.1 ms
      Minibatch loss: 3.351, learning rate: 0.010000
      

      • tensorflow cpu only:

      I tensorflow/models/embedding/word2vec_kernels.cc:200] Data file: data/text8 contains 100000000 bytes, 17005207 words, 253854 unique words, 71290 unique frequent words.
      Data file:  data/text8
      Vocab size:  71290  + UNK
      Words per epoch:  17005207
      mu_scale = 0.2449489742783178 var_scale = 0.05
      

5. make nvida docker runnning tensorflow gpu
   

   sudo apt-get install -y nvidia-docker2=2.0.2+docker1.13.1 nvidia-container-runtime=1.1.1+docker1.13.1
   

1.2.1 Shell basics

TEST_IMG=ADE_val_00001514.jpg
MODEL_PATH=baseline-resnet50_dilated8-ppm_bilinear_deepsup
RESULT_PATH=./

ENCODER=$MODEL_PATH/encoder_epoch_20.pth
DECODER=$MODEL_PATH/decoder_epoch_20.pth
CONTENT=images/content
STYLE=images/style

if [ ! -e $ENCODER ]; then
  mkdir $MODEL_PATH
fi
if [ ! -e $ENCODER ]; then
  wget -P $MODEL_PATH http://sceneparsing.csail.mit.edu/model/pytorch/$ENCODER
fi
if [ ! -e $DECODER ]; then
  wget -P $MODEL_PATH http://sceneparsing.csail.mit.edu/model/pytorch/$DECODER
fi
if [ ! -e $TEST_IMG ]; then
  wget -P $RESULT_PATH http://sceneparsing.csail.mit.edu//data/ADEChallengeData2016/images/validation/$TEST_IMG
fi

for f in $(ls $CONTENT); do
  echo $f
 python3 -u test.py \
  --model_path $MODEL_PATH \
  --test_img $CONTENT/$f \
  --arch_encoder resnet50_dilated8 \
  --arch_decoder ppm_bilinear_deepsup \
  --fc_dim 2048 \
  --result $STYLE

done

/bin/bash
The bash executable
/etc/profile
The systemwide initialization file, executed for login shells
~/.bash_profile
The personal initialization file, executed for login shells
~/.bashrc
The individual per-interactive-shell startup file
~/.bash_logout
The individual login shell cleanup file, executed when a login shell exits
~/.inputrc
Individual readline initialization file

vim /etc/profile

exec $SHELL -l
source ~/.zshrc

-r, --reverse
reverse order while sorting

-S     sort by file size

--sort=WORD
sort by WORD instead of name: none -U, extension -X, size -S, time -t, version -v

-t     sort by modification time, newest first

tail -f -n 100 wiki_upload.log

realpath filename

xev

ls -1 | wc -l
# count all the files recursively
find DIR_NAME -type f | wc -l

md5sum filename

exec "$SHELL"

alias lf="ls -l | egrep -v '^d'"
alias ldir="ls -l | egrep '^d'"

find ./ -type d

rm -R -- */

#!/bin/bash
show_file()
{
    for file in `ls $1`
    do
        if [ -d $1"/"$file ];then
            #目录递归调用show_file函数
            show_file $1"/"$file
        else
            #文件
            echo "文件繁体转简体"$1"/"$file
            opencc -i $1"/"$file -o $1"/"$file".zhs" -c zht2zhs.ini
            rm $1"/"$file
        fi
    done
}
path=./zhswiki
show_file $path

# to rename all files matching "*.bak" to strip the extension
rename 's/\.bak$//' *.bak

1.2.2 bash scripting

1.2.3 Regular expressions

1.2.4 Perl programming

1.2.5 Ruby

sudo gem install ttfunk

# Get the cloned pdf-reader-1.3.3 (and uninstall all other versions)
git clone https://github.com/yob/pdf-reader
cd pdf-reader
gem build pdf-reader.gemspec
sudo gem install pdf-reader-1.3.3.gem # check version number

# Then get the cloned pdf-extract (not the same as zip)
git clone https://github.com/CrossRef/pdfextract
cd pdfextract
gem build pdf-extract.gemspec
sudo gem install pdf-extract-0.1.1.gem

1.2.6 Python scripting

1.2.7 Scripting best practices

1.2.8 Shell commands:

reset

pwd |  xclip -i -selection clipboard

cp folder /destination/path -a

uname -m && cat /etc/*release

soffice filename

scp usrname@ip:/home/weiwu/share/deep_learning/data/enwiki-20170820-pages-articles.xml.bz2 ./

scp file usr_name@ip:/absolute/path/
# copy folder
scp -r ./path/from/destination username@hostname:/path/to/destination

-P pattern
      List  only  those files that match the wild-card pattern.  Note:
      you must use the -a option to also consider those  files  begin‐
      ning  with a dot `.' for matching.  Valid wildcard operators are
      `*' (any zero or more characters), `?' (any  single  character),
      `[...]'  (any single character listed between brackets (optional
      - (dash) for character  range  may  be  used:  ex:  [A-Z]),  and
      `[^...]'  (any  single character not listed in brackets) and `|'
      separates alternate patterns.

-I pattern
      Do not list those files that match the wild-card pattern.
So, for example

tree -I 'test*|docs|bin|lib'

tree -P '*.org' | xclip && xclip -o > /tmp/tree.txt
tree -P '*.sh|*.py|*.csv|*.ddlog|*.url|*.conf' | xclip && xclip -o > /tmp/tree.txt
tree | xclip
xclip -o

wc -l filename

ls -l folder/path | wc -l

wget -c -t 0 --timeout=10 --waitretry=10 url
# zsh: parse error near &
wget “<DOWNLOAD_URL>”

axel -n 10 url
unzip *.zip -d folder/

du -sh *

# du only for directories
du -sh ./*/

unzip filename.zip
# batch unzip
ls *.zip | xargs -n1 unzip -o -P infected

# tar.gz
tar -xvzf rebol.tar.gz
tar -xvjSf *.tar.bz2
# batch extract
ls *.tar.gz | xargs -n1 tar xzvf
简介

参数：

-c  ：建立一个压缩档案的参数指令(create 的意思)；

-x  ：解开一个压缩档案的参数指令！

-t  ：查看 tarfile 里面的档案！

特别注意，在参数的下达中， c/x/t 仅能存在一个！不可同时存在！

因为不可能同时压缩与解压缩。

-z  ：是否同时具有 gzip 的属性？亦即是否需要用 gzip 压缩？

-j  ：是否同时具有 bzip2 的属性？亦即是否需要用 bzip2 压缩？

-v  ：压缩的过程中显示档案！这个常用，但不建议用在背景执行过程！

-f  ：使用档名，请留意，在 f 之后要立即接档名喔！不要再加参数！
#*.gz

sudo service network-manager restart

　　- rw- r-- r--

sudo chmod -R 0777 /folder

ln -s /absolute/path/of/source/folder/ /absolute/path/of/target/folder/
# create a link dogscats folder in projects/deep_learning/basic/courses/deeplearning1/nbs/data/
ln -s /home/weiwu/share/deep_learning/data/courses/dogscats/ /home/weiwu/projects/deep_learning/basic/courses/deeplearning1/nbs/data/

1.2.9 find/search string

sed -i 's/old-text/new-text/g' input.txt

less +320123 filename

# grep = global regular expression parser
grep -inR "Regular Expression" .
grep -inR "Regular Expression" . | less
如果有符号{},要在前面加\, \{2,5\}
# find specific formats
grep -inR "Regular Expression" . --include \*.org

# search for gid
grep '[0-9A-Z]\{32\}' wiki_upload.log > log.txt

ps -ef | grep mysql

awk '!seen[$0]++' filename

sudo find / -type f -executable -iname 'python*' -exec file -i '{}' \; | awk -F: '/x-executable; charset=binary/ {print $1}' | xargs readlink -f | sort -u | xargs -I % sh -c 'echo -n "%: "; % -V'

# 2
ls -ls /usr/bin/python*

grep "_" vocabulary > phrase_vocab

find . -name "name*"
find . -path '*/folder/subfolder'

1.2.10 zsh

# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

# Path to your oh-my-zsh installation.
  export ZSH=/home/weiwu/.oh-my-zsh

# Set name of the theme to load. Optionally, if you set this to "random"
# it'll load a random theme each time that oh-my-zsh is loaded.
# See https://github.com/robbyrussell/oh-my-zsh/wiki/Themes
ZSH_THEME="robbyrussell"
# ZSH_THEME="agnoster"
# ZSH_THEME="pygmalion"

# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"

# Uncomment the following line to use hyphen-insensitive completion. Case
# sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"

# Uncomment the following line to disable bi-weekly auto-update checks.
# DISABLE_AUTO_UPDATE="true"

# Uncomment the following line to change how often to auto-update (in days).
# export UPDATE_ZSH_DAYS=13

# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"

# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="true"

# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"

# Uncomment the following line to display red dots whilst waiting for completion.
# COMPLETION_WAITING_DOTS="true"

# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
# DISABLE_UNTRACKED_FILES_DIRTY="true"

# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# The optional three formats: "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
# HIST_STAMPS="mm/dd/yyyy"
SAVEHIST=1000000
HISTFILE=~/.zsh_history

# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder

# Which plugins would you like to load? (plugins can be found in ~/.oh-my-zsh/plugins/*)
# Custom plugins may be added to ~/.oh-my-zsh/custom/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git jump z)

source $ZSH/oh-my-zsh.sh

# User configuration

# export MANPATH="/usr/local/man:$MANPATH"

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# ssh
# export SSH_KEY_PATH="~/.ssh/rsa_id"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
# alias ohmyzsh="mate ~/.oh-my-zsh"
export XMODIFIERS="@im=fcitx"
export GTK_IM_MODULE=xim
export QT_IM_MODULE=xim
export LC_CTYPE=zh_CN.UTF-8
alias lf="ll -l | egrep -v '^d'"
alias ldir="ll -l | egrep '^d'"
alias lwc="ll -1 | wc -l"
#alias got="ps awx | grep"
alias got="realpath "$1" |  xclip -i -selection clipboard"
function got()
{
    readlink -f $1 | xclip -i -selection clipboard
}
# path
export PATH=$HOME/Tools/google_appengine:$PATH
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

# emacs
alias ec='emacsclient -nc -a ""'
export EDITOR="emacsclient"

# cpp
export CPLUS_INCLUDE_PATH=$HOME/github/rlib-cpp:$HOME/github/ib-tws-trade:$HOME/github/ib-tws-trade/PosixClient/Shared:$CPLUS_INCLUDE_PATH
export GTAGSLIBPATH=$HOME/.gtags/
export AG_STRATEGY=$HOME/projects/ag-strategy:$AG_STRATEGY

# node
alias npm='npm --registry=https://registry.npm.taobao.org'

# python
#export PYENV_ROOT="$HOME/.pyenv"

export PYTHONPATH=/usr/local/lib/python2.7/:$PYTHONPATH
# export PYTHONPATH=$HOME/projects/tradehub/src/py/:$PYTHONPATH
export PYTHONPATH=$HOME/projects/deep_learning/:$PYTHONPATH
export PYTHONPATH=$HOME/projects/simulate/:$PYTHONPATH
export PYTHONPATH=$HOME/tools/WaveFront/PythonScripts/:$PYTHONPATH
export PYTHONPATH=$HOME/projects/ylib-py/:$PYTHONPATH
export PYTHONPATH=$HOME/projects/:$PYTHONPATH
export PYENV_VIRTUALENVWRAPPER_PREFER_PYVENV="true"

# for file in `find /home/weiwu/tools/stanford-corenlp-full-2017-06-09/ -name "*.jar"`; do export
# CLASSPATH="$CLASSPATH:`realpath $file`"; done
# export STANFORD_SEGMENTER_PATH="$HOME/tools/segmenter"
# #export CLASSPATH="$CLASSPATH:$STANFORD_SEGMENTER_PATH/stanford-segmenter.jar"
# export STANFORD_NER_PATH="$HOME/tools/tools/stanford-ner-2017-06-09"
# export CLASSPATH="$CLASSPATH:$STANFORD_NER_PATH/stanford-ner.jar"
#export PATH="$PYENV_ROOT/bin:$PATH"

#pyenv virtualenvwrapper

#export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python

source /usr/local/bin/virtualenvwrapper.sh
export PYENV_ROOT=$HOME/.pyenv
export PATH=$PYENV_ROOT/bin:$PATH
eval "$(pyenv init -)"

#export PATH="$PYENV_ROOT/bin:$PATH
#eval "$(pyenv init -)"
#pyenv virtualenvwrapper

export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python


# The next line updates PATH for the Google Cloud SDK.
if [ -f '/home/weiwu/projects/sentiment/google-cloud-sdk/path.zsh.inc' ]; then source '/home/weiwu/projects/sentiment/google-cloud-sdk/path.zsh.inc'; fi

# The next line enables shell command completion for gcloud.
if [ -f '/home/weiwu/projects/sentiment/google-cloud-sdk/completion.zsh.inc' ]; then source '/home/weiwu/projects/sentiment/google-cloud-sdk/completion.zsh.inc'; fi
fpath=(~/.zsh/completion $fpath)
autoload -Uz compinit && compinit -i

1.2.11 make cmake compile

1. cmake
   

   http://derekmolloy.ie/hello-world-introductions-to-cmake/

   1. Example 2: A Project with Directories
      

      molloyd@beaglebone:~/exploringBB/extras/cmake/student$ tree
      .
      |-- CMakeLists.txt
      |-- build
      |-- include
      |   \-- Student.h
      \-- src
          |-- Student.cpp
          \-- mainapp.cpp
      3 directories, 4 files
      

      cd build
      cmake ..
      make
      ls
      ./test
      

1.2.12 update/install git

1.3.1 Bootstrapping

1.3.2 Booting PCs

1.3.3 GRUB: The GRand Unified Boot loader

1.3.4 Booting to single-user mode

1.3.5 Working with startup scripts

1.3.6 Booting Solaris

1.3.7 Rebooting and shutting down

1.4.1 Traditional UNIX access control

1.4.2 Modern access control

1.4.3 Real-world access control

1.4.4 Pseudo-users other than root

1.5.1 Components of a process

1.5.2 The life cycle of a process

1.5.3 Signals

1.5.4 kill: send signals

pkill -f process_name
# or
ps aux | grep -ie amarok | awk '{print $2}' | xargs kill -9

1.5.5 Process states

1.5.6 nice and renice: influence scheduling priority

1.5.7 ps: monitor processes

1.5.8 Dynamic monitoring with top, prstat, and topas

1.5.9 The /proc filesystem

1.5.10 strace, truss, and tusc: trace signals and system calls

1.5.11 Runaway processes

1.5.12 Check Memory usage:

cat /proc/meminfo # 或者 更直观的查看内存的命令：free -m
free -m
top -H

1.6.1 Pathnames

readlink -f file.txt

1.6.2 Filesystem mounting and unmounting

1.6.3 The organization of the file tree

1.6.4 File types

1.6.5 File attributes

1.6.6 Access control lists

cat 'file' | sed -n '25p'

zip -r filename.zip target_dir
tar -cvjSf folder.tar.bz2 folder

1.7.1 The /etc/passwd file

1.7.2 The /etc/shadow and /etc/security/passwd files

1.7.3 The /etc/group file

1.7.4 Adding users: the basic steps

1.7.5 Adding users with useradd

1.7.6 Adding users in bulk with newusers (Linux)

1.7.7 Removing users

sudo userdel -r username

1.7.8 Disabling logins

1.7.9 Managing users with system-specific tools

1.7.10 Reducing risk with PAM

1.7.11 Centralizing account management

1.8.1 I just want to add a disk!

1. add a new disk:
   

   fdisk -l
   fdisk /dev/hdd
   # （选中对新硬盘/dev/hdd进行操作）
   Command (m for help): d
   1-end
   
   p （p表示新建主分区）
   .Command (m for help): w（完成分区规划后不要忘记用w命令保存退出，如果觉得之前的分区工作不合理，千万不要使用w退出，使用q不保存退出便可以重新进行磁盘划分）
   
   mkfs.ext3 /dev/hdd1
   (注：需要手动对每个分区进行格式化，不能一次性格式整个硬盘，例如:mkfs.ext3 /dev/sda，否则会格式化失败。)
   
   mount /dev/hdd1 /home/oem/extended(将主分区1挂载到/home/oem/extended目录)
   df -h
   (查看新硬盘挂载后的信息，确保正确无误)
   
   sudo gedit /etc/fstab
   编辑fstab文件
   （这点十分重要，一定要记得编辑该文件，不然重启电脑后系统将不会保存之前的挂载操作）
   /dev/hdd5            /usr/local/superman    ext3    defaults        1 1
   
   
   

1.8.2 Storage hardware

# human readable
df -H
sudo fdisk -l

df -sh *

1.8.3 Storage hardware interfaces

1.8.4 Peeling the onion: the software side of storage

1.8.5 Attachment and low-level management of drives

1.8.6 Disk partitioning

1.8.7 RAID: redundant arrays of inexpensive disks

1.8.8 Logical volume management

1.8.9 Filesystems

1. give user permission to write a folder
   

   # change the user owning this file or directory (folder)
   sudo chown username: myfolder
   # to add the write permission to the username user
   sudo chmod u+w myfolder
   

1.8.10 ZFS: all your storage problems solved

1.8.11 Storage area networking

1.9.1 cron: schedule commands

1.9.2 The format of crontab files

1.9.3 Crontab management

1.9.4 Linux and Vixie-cron extensions

1.9.5 Some common uses for cron

1.10.1 Motherhood and apple pie

1.10.2 Backup devices and media

1.10.3 Saving space and time with incremental backups

1.10.4 Setting up a backup regime with dump

1.10.5 Dumping and restoring for upgrades

1.10.6 Using other archiving programs

1.10.7 Using multiple files on a single tape

1.10.8 Bacula

1.10.9 Commercial backup products

1.11.1 Finding log files

1.11.2 Syslog: the system event logger

1.11.3 AIX logging and error handling

1.11.4 logrotate: manage log files

1.11.5 Condensing log files to useful information

1.11.6 Logging policies

1.12.1 Installing Linux and OpenSolaris

1.12.2 Installing Solaris

1.12.3 Installing HP-UX

1.12.4 Installing AIX with the Network Installation Manager

1.12.5 Managing packages

1.12.6 Managing Linux packages

1.12.7 Using high-level Linux package management systems

1.12.8 Managing packages for UNIX

1.12.9 Revision control

1.12.10 Software localization and configuration

1.12.11 Using configuration management tools

1.12.12 Sharing software over NFS

sudo apt-get install nfs-common rpcbind

sudo apt-get install nfs-kernel-server rpcbind

mkdir -p /export/users

sudo gedit /etc/fstab
/home/users    /export/users   none    bind  0  0

gksu gedit /etc/exports
/home/weiwu/Documents/projects           192.168.1.73(rw,sync,no_root_squash,no_subtree_check)

sudo service nfs-kernel-server restart

sudo exportfs -a
# verify
sudo exportfs -v

sudo mount -t nfs4 -o proto=tcp,port=2049 192.168.1.199:/home/weiwu/Documents/projects /home/oem/projects/share

1.13.1 Kernel adaptation

1.13.2 Drivers and device files

1.13.3 Linux kernel configuration

1.13.4 Solaris kernel configuration

1.13.5 HP-UX kernel configuration

1.13.6 Management of the AIX kernel

1.13.7 Loadable kernel modules

1.13.8 Linux udev for fun and profit

1.14.1 notepad ++

1. Use the Search-->Find-->Mark functionality to mark each line you want to keep/copy and remember to tick 'Bookmark Line' before marking the text
2. Select Search-->Bookmark-->Remove Unmarked Lines
3. Use Search-->Find-->Replace to replace other text you do not want to keep/copy with nothing
4. Save the remaining text or copy it.

1.14.2 nano

1.14.3 emacs

2.1.1 TCP/IP and its relationship to the Internet

2.1.2 Networking road map

2.1.3 Packet addressing

2.1.4 IP addresses: the gory details

# Ubuntu 14.04
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

auto eth1
iface eth1 inet static
address 192.168.1.128
netmask 255.255.255.0
network 192.168.1.1
dns-nameservers 8.8.8.8

auto eth0
iface eth0 inet static
address 192.168.1.199
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8

# Ubuntu 16.04
sudo gedit /etc/network/interfaces
auto eno1
iface eno1 inet static
address 192.168.1.73
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8

2.1.5 Routing

2.1.6 ARP: the Address Resolution Protocol

2.1.7 DHCP: the Dynamic Host Configuration Protocol

sudo gedit /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback


auto eth0
iface eth0 inet static
address 192.168.1.199
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet static
address 192.168.1.128
netmask 255.255.255.0
network 192.168.1.1
dns-nameservers 8.8.8.8

sudo ifdown eth0 && sudo ifup eth0
sudo service networking restart

2.1.8 Security issues

2.1.9 PPP: the Point-to-Point Protocol

2.1.10 Basic network configuration

2.1.11 System-specific network configuration

2.1.12 Linux networking

2.1.13 Solaris networking

2.1.14 HP-UX networking

2.1.15 AIX networking

2.2.1 Packet forwarding: a closer look

2.2.2 Routing daemons and routing protocols

2.2.3 Protocols on parade

2.2.4 Routing strategy selection criteria

2.2.5 Routing daemons

2.2.6 Cisco routers

2.3.1 Ethernet: the Swiss Army knife of networking

2.3.2 Wireless: ethernet for nomads

2.3.3 DSL and cable modems: the last mile

2.3.4 Network testing and debugging

2.3.5 Building wiring

2.3.6 Network design issues

2.3.7 Management issues

2.3.8 Recommended vendors

2.4.1 Who needs DNS?

2.4.2 How DNS works

2.4.3 DNS for the impatient

2.4.4 Name servers

2.4.5 The DNS namespace

2.4.6 Designing your DNS environment

2.4.7 What’s new in DNS

2.4.8 The DNS database

2.4.9 The BIND software

2.4.10 BIND configuration examples

2.4.11 The NSD/Unbound software

2.4.12 Updating zone files

2.4.13 Security issues

2.4.14 Microsoft and DNS

2.4.15 Testing and debugging

2.4.16 Vendor specifics

2.5.1 Introduction to network file services

2.5.2 The NFS approach

2.5.3 Server-side NFS

2.5.4 Client-side NFS

2.5.5 Identity mapping for NFS version 4

2.5.6 nfsstat: dump NFS statistics

2.5.7 Dedicated NFS file servers

2.5.8 Automatic mounting

# mount a windows shared folder
sudo mount -t cifs -o username="gft" //192.168.1.126/share ~/share

2.6.1 What to share

2.6.2 Copying files around

2.6.3 LDAP: the Lightweight Directory Access Protocol

2.6.4 NIS: the Network Information Service

2.6.5 Prioritizing sources of administrative information

2.7.1 Mail systems

2.7.2 The anatomy of a mail message

2.7.3 The SMTP protocol

2.7.4 Mail system design

2.7.5 Mail aliases

2.7.6 Content scanning: spam and malware

2.7.7 Email configuration

2.7.8 sendmail

2.7.9 sendmail configuration

2.7.10 sendmail configuration primitives

2.7.11 Security and sendmail

2.7.12 sendmail performance

2.7.13 sendmail testing and debugging

2.7.14 Exim

2.7.15 Postfix

2.7.16 DKIM Configuration

2.7.17 Integrated email solutions

2.8.1 Network troubleshooting

2.8.2 ping: check to see if a host is alive

2.8.3 SmokePing: gather ping statistics over time

2.8.4 traceroute: trace IP packets

2.8.5 netstat: get network statistics

netstat -na -f inet | grep LISTEN
# example netstat -na -f inet | grep 5277

2.8.6 Inspection of live interface activity

2.8.7 Packet sniffers

2.8.8 The ICSI Netalyzr

2.8.9 Network management protocols

2.8.10 SNMP: the Simple Network Management Protocol

2.8.11 The NET-SNMP agent

2.8.12 Network management applications

2.8.13 NetFlow: connection-oriented monitoring

tcpdump ip proto \\icmp

2.9.1 Is UNIX secure?

2.9.2 How security is compromised

2.9.3 Security tips and philosophy

2.9.4 Passwords and user accounts

2.9.5 PAM: cooking spray or authentication wonder?

2.9.6 Setuid programs

2.9.7 Effective use of chroot

2.9.8 Security power tools

2.9.9 Mandatory Access Control (MAC)

2.9.10 Cryptographic security tools

2.9.11 Firewalls

2.9.12 Linux firewall features

2.9.13 IPFilter for UNIX systems

2.9.14 Virtual private networks (VPNs)

2.9.15 Certifications and standards

2.9.16 Sources of security information

2.9.17 What to do when your site has been attacked

2.10.1 Web hosting basics

2.10.2 HTTP server installation

2.10.3 Virtual interfaces

2.10.4 The Secure Sockets Layer (SSL)

2.10.5 Caching

2.10.6 chrome cookies

3.1.1 Virtual vernacular

1. screen
   

   Screen or GNU Screen, is a terminal multiplexer. In other words, it means that you can start a screen session and then open any number of windows(virtual terminals) inside that session. Processes running in Screen will continue to run when their window is not visible even if you get disconnected.

   1. installation
      

      yum install screen
      # or
      sudo apt-get install screen
      

   2. start a linux screen
      

      screen -S session_name
      Ctrl+a c # Create a new window (with shell)
      Ctrl+a Ctrl+a # Toggle between current and previous region
      
      

   3. Detach from Linux Screen Session
      

      Ctrl+a d

   4. Reattach to a Linux Screen
      

      screen -ls
      screen -r session
      
      

3.1.2 Benefits of virtualization

3.1.3 A practical approach

3.1.4 Virtualization with Linux

3.1.5 Solaris zones and containers

3.1.6 AIX workload partitions

3.1.7 Integrity Virtual Machines in HP-UX

3.1.8 VMware: an operating system in its own right

3.1.9 Amazon Web Services

3.2.1 The display manager

3.2.2 Process for running an X application

3.2.3 X server configuration

3.2.4 X server troubleshooting and debugging

3.2.5 A brief note on desktop environments

3.3.1 Printing-system architecture

3.3.2 CUPS printing

3.3.3 Printing from desktop environments

3.3.4 System V printing

3.3.5 BSD and AIX printing

3.3.6 What a long, strange trip it’s

3.3.7 Common printing software

3.3.8 Printer languages

3.3.9 PPD files

3.3.10 Paper sizes

3.3.11 Printer practicalities

3.3.12 Troubleshooting tips

3.4.1 Data center reliability tiers

3.4.2 Cooling

3.4.3 Power

3.4.4 Racks

3.4.5 Tools

3.5.1 Green IT initiation

3.5.2 The green IT eco-pyramid

3.5.3 Green IT strategies: data center

3.5.4 Green IT strategies: user workspace

3.5.5 Green IT friends

3.6.1 What you can do to improve performance

3.6.2 Factors that affect performance

3.6.3 How to analyze performance problems

3.6.4 System performance checkup

3.6.5 Help! My system just got really slow!

3.7.1 Logging in to a UNIX system from Windows

3.7.2 Accessing remote desktops

sudo apt-get install vinagre -y

3.7.3 Running Windows and Windows-like applications

3.7.4 Using command-line tools with Windows

3.7.5 Windows compliance with email and web standards

3.7.6 Sharing files with Samba and CIFS

3.7.7 Sharing printers with Samba

3.7.8 Debugging Samba

3.7.9 Active Directory authentication

3.8.1 The RS-232C standard

3.8.2 Alternative connectors

3.8.3 Hard and soft carrier

3.8.4 Hardware flow control

3.8.5 Serial device files

3.8.6 setserial: set serial port parameters under Linux

3.8.7 Pseudo-terminals

3.8.8 Configuration of terminals

3.8.9 Special characters and the terminal driver

3.8.10 stty: set terminal options

3.8.11 tset: set options automatically

3.8.12 Terminal unwedging

3.8.13 Debugging a serial line

3.8.14 Connecting to serial device consoles

3.9.1 start a docker

service docker start

3.9.2 manage a docker image

3.9.3 go inside a docker

docker exec -it container的ID /bin/bash