========= Linux OS ========= => We will install devops tools in linux machines only a) Jenkins Server b) Docker s/w c) k8s cluster d) sonarqube server e) nexus server f) ELK stack g) Ansible => database server will be installed in linux machine => application deployment will happen in linux machine Note: To become DevOps engineer we should have strong knowledge in Linux OS. ============= What is OS ? ============= => It is a software which acts as mediator between users and computers => Users will communicate with computers using OS s/w. => Without OS we can't use any computer. => OS will provide platform to run our applications in computer. Ex: notepad, ms paint, calculator, browser... => We have several Operating systems in the market Ex: Windows, Linux, Mac, Android, IOS etc... ============ Windows OS ============ => Developed By Microsoft company (Bill Gates) => Windows OS is licensed software (commercial) => Windows is single user based OS => Security Features are very less in windows (anti virus s/w required) => Windows is GUI based (Graphical User Interface) => Windows OS is recommended for personal computers Ex: watch movie, play games, ppt, online classes ========= Linux OS ========= => Linux is community based OS (not specific to any company) => Linux is free and open source Operating system (OSS) => Linux is multi user based OS => Linux is Highly secured (anti virus not required) => Linux supports both GUI and CLI CLI : Commandline interface => Linux is highly recommended for business use cases Ex: App servers, DB servers, DevOps tools setup etc... ============== Linux History ============== => Linux OS developed by "Linus Torvalds" => Linus Torvalds identified some issues with Unix OS. He suggested changes to Unix OS but that company not accepted. => Linus Torvalds identified "Minux OS" having similiar features what he is expecting. => Linus Torvalds downloaded "Minux OS code" and modified according to his ideas and released into market as 'Linux OS'. (Li)nus + Mi(nux) = Linux ==================== Linux Distributions ==================== => Linus Torvalds provided Linux OS source code for free of cost. => So many companies downloaded linux os code and modified according to their requirement and released into market with different names those are called as Linux Distributions. => We have 200+ Linux distributions in the market Ex: Amazon Linux, Ubuntu, Cent OS, Red HAT, Kali, SUSE... ============================== How to setup Linux Machine ? ============================== Option-1 :: Download and install Linux OS in your machine Option-2 :: Install Linux OS as Guest OS using Virtual Box Option-3 :: Setup Linux machine in cloud (Virtual Machine) Note: When we setup Linux Virutal Machine in AWS cloud, we can connect with that VM using SSH client softwares. ex : Git Bash, MobaXterm, Putty .. ================================ Linux Architecture Components =============================== 1) Applications / Commands 2) Shell 3) Kernel 4) Hardware components =================== What is shell ? =================== => Shell acts as mediator between user and kernel. => Shell is responsible to process user given commands. Note: when we execute a command, shell verify command syntax. If commad is valid then shell will convert that command into kernel understable format. # check default shell of our linux vm echo $SHELL ========================== What is Kernel in linux ? ========================== => Kernel is heart of Linux OS => Kernel is a mediator between SHELL and Hardware components. => Kernel will get instructions from shell then kernel will convert that command into hardware understandable format. # print kernel version uname -r =================== Linux File System =================== => In Linux OS everything will be represented as File only. => Root Directory (/) is starting point of linux machine => Inside root directory we have several directories like below /home : It contains user home directories Note: For every user one home directory will be available like below. ec2-user : /home/ec2-user/ ashok : /home/ashok/ raju : /home/raju/ /bin : Binaries will be available (linux os programs) /boot : static of boot loader /dev : device files /lib : Shared libraries /etc : Host specific configuration files. /opt : Optional application software packages /tmp : Temporary files will be stored here /usr : user utilities and applications /mnt : Mounted file system /media : Removable media files ================= Linux Commands ================= whoami : Display logged-in username pwd : Display present working directory date : Display current date cal : Display current month calendar cal 2030 : Display 2030 calendar cd : change directory cd .. (go one step back from pwd) cd : Go inside directory mkdir : Make directory (create folder) mkdir linux mkdir aws mkdir devops azure gcp rmdir : remove empty directory (delete) rmdir aws ls : display present working directory content ls -l : long list the files in alphabetical order ls -lr : display files in reverse of alphabetical order ls -lt : display latest files on top ls -ltr : display old files on top ls -la : display hidden files touch : to create empty files in linux rm : to delete file rm f1.txt rm -rf (to delete non empty directories) mv : For rename and move files & directories mv existing-name new-name mv presention-location new-location cat : create new file with data + append data to file + print file data # create new file with data cat > f1.txt # append data to existing file cat >> f1.txt # print file data from top to bottom cat f1.txt # print file data with line numbers cat -n f1.txt tac : To print file data from bottom to top tac f1.txt cp : copy data from one file to another file cp f1.txt f2.txt Note: If we want to copy data from multiple files then we should use cat command cat f1.txt f2.txt > f3.txt head : print first 10 lines of the file head f1.txt #print first 5 lines of data head -n 5 f1.txt # print first 30 lines of data head -n 30 f1.txt tail : print last 10 lines of the file tail f1.txt # print last 25 lines tail -n 25 f1.txt grep : Global Regular expression print Note: Using grep command we can search for content in the file # print lines which contains java keyword grep 'java' fullstack.txt # ignore case-sensitive grep -i 'java' fullstack.txt # print lines which doesn't contains java keyword (invert match) grep -v 'java' fullstack.txt # search for java keyword in last 10 lines of file # tail with grep combination tail fullstack.txt | grep 'java' wc : word count # print no.of lines, no.of words, no.of letters in the file wc fullstack.txt ======================= Text Editors in Linux ======================= => vi (visual editor) it is default editor in linux machines. => Using 'vi' we can create new files and we can modify existing file data. => vi command is having 3 modes 1) command mode (just to open the file) 2) insert mode (to edit the file) ----> press 'i' in keyboard 3) esc mode (to comeout from insert mode) --> press 'esc' in keyword # save the changes and close the file ====> :wq + enter # close the file without saving changes ===> :q! + enter Note: vi command will open the file if it is already avilable otherwise it will create new file and it will open that file. ============= SED Command ============= => SED stands for stream editor => SED is used to process the data (substitute, delete, print, insert) => Using SED command we can perform operations on the file without opening the file. => we will use below options with SED command s - substitute d - delete p - print i - insert # replace first occurance of linux with unix in every line sed 's/linux/unix/' os.txt # replace second occurance of linux with unix in every line sed 's/linux/unix/2' os.txt # replace third occurance of linux with unix in every line sed 's/linux/unix/3' os.txt # replace all occurances of linux keyword with unix sed 's/linux/unix/g' os.txt # substitute linux with unix and save changes in the file sed -i 's/linux/unix/g' os.txt # delete first line of the file sed -i '1d' os.txt # delete 4th line of the file sed -i '4d' os.txt # delete last line of the file sed -i '$d' os.txt # delete from 3rd line to last line sed -i '3, $d' os.txt # delete from 10th line to 15th line sed -i '10, 15d' os.txt # Add line at end of file sed -i '$a\i am from ashokit' os.txt # Add line at end of file sed -i '2i\i want to learn devops' os.txt # print 10th line only sed -n '10p' os.txt # print data from 2nd line to 5th line sed -n '2,5p' os.txt ================================= Working with Zip files in linux ================================= => Zip is used for files archieve (compress) ## syntax to create zip file : $ zip # create zip with all .txt files zip myfiles *.txt # extract zip file unzip myfiles.zip ====================== Networking commands ====================== ping : To check connectivity ping www.google.com ping www.facebook.com ping 102.18.4.1 wget : It is used to download files from internet wget curl : It is used to send http request to server curl https://dummyjson.com/quotes/random ifconfig : To get IP address of our machine ===================== process management ===================== # display running processes with PID $ ps aux Note: Every process will have process id (PID) # kill process kill # terminate process immediatley (forcefully) kill -9 ================= User Management ================= => Linux is a multi user based OS => Multiple users can acces single linux machine and can perform multi tasking at time. Note: "ec2-user" is a default user in "amazon linux" vm. ec2-user having sudo priviliges. Amazon Linux AMI ===> ec2-user (default user) Ubuntu AMI ==> ubuntu (default user) => when we create user account, for user one home directory will be created. ec2-user => /home/ec2-user/ john => /home/john/ smith => /home/smith/ # display all users created cat /etc/passwd # create user sudo useradd # set pwd for user sudo passwd # swith user account su # navigate to current user home directory cd ~ # delete user along with user home directory sudo userdel --remove /etc/passwd : This file contains user information /etc/shadow : This file contains user passwords in encrypted format =================================== Working with user groups in linux =================================== => When we create user in linux, for that user one user group also will be created with the given username. # display all groups cat /etc/group # create group in linux sudo groupadd # add user to group sudo usermod -aG # remove user from group sudo gpasswd -d # display users belongs to group sudo lid -g # check one user belongs to how many groups id # delete group sudo groupdel ======================================================================================= Assignment : create new user and connect with linux vm using newly created user account ======================================================================================= => To connect with Linux VM using our own user account (Ex: ashokit) , we need to enable Password Based Authentication in Linux VM. => To enable PasswordBasedAuthentication in linux vm then we need to modify below configuration files 1) sudoers 2) sshd_config ================================= What is sudoers file in Linux ================================= => It is very important configuration file in linux machine. => Using this file we can control which user can run command as a superuser. # print sudoers file content sudo cat /etc/sudoers # Open sudoers file sudo visudo # Add below line under root user ALL=(ALL) ALL => After making the changes, to close sudoers file we need to execute => ( CTRL + X + Y + Enter ) ===================================================================== How to enable password based authentication for users in linux vm ? ===================================================================== => In linux vm, by default passwordauthentication is no => If we want to connect with linux vm by using username and password then we need to set passwordauthentication value as yes. => WE WLL MODIFY THIS IN "sshd_config" file. # Display sshd_configurration file data $ sudo cat /etc/ssh/sshd_config # Open file $ sudo vi /etc/ssh/sshd_config Note: Go to insert mode and enable pwdbasedauthentication as yes # restart sshd service # sudo systemctl restart sshd Note : Now we can connect with linux vm using username and pwd Step-1 : Open gitbash Step-2 : Execute below command $ ssh uname@linux-vm-public-ip ========================== File Permissions in Linux ========================== => Using file permissions we can secure our files and we can protect our file data. => We have 3 types of permissions in linux r => read w => write x => execute => file/directory permissions will be represented like below rwxrwxrwx f1.txt => file permissions contains 9 characters and those are divided into 3 parts first 3 characters => user/owner permissions middle 3 characters => group permissions last 3 characters => other users permissions => Lets under file permissions with diff scenarios Scenario-1 :: rw-r--r-- f1.txt user : (rw-) : read + write group : (r--) : read others : (r--) : read Scenario-2 :: rwxr-xr-x f1.txt user : (rwx) : read + write + execute group : (r-x) : read + execute others: (r-x) : read + execute Scenario-3 : rwx--xr-x f1.txt user : ( rwx ) : read + write + execute group : ( --x ) : execute others : ( r-x ) : read + execute => To change file/directory permissions we will use 'chmod' command + represents adding - represents removing # Giving execute permission for user chmod u+x f1.txt # giving write permission for group chmod g+w f1.txt # Remove execute permission for others chmod o-x f1.txt # give all permissions for group chmod g+rwx f1.txt # Remove all permissions for others chmod o-rwx f1.txt ==================================== File Permissions in Numeric Format ==================================== 0 => No Permission 1 => Execute 2 => Write 3 => (2+1) => Write + Execute 4 => Read 5 => (4 + 1) => Read + Execute 6 => (4 + 2) => Read + Write 7 => (6 + 1) => Read + Write + Execute chmod 111 f1.txt chmod 222 f1.txt chmod 007 f1.txt chmod 077 f1.txt =============== chown command =============== => It is used to change file/directory ownership # change owner sudo chown # change owner-group sudo chown :new-group # change owner & group of file/directory sudo chown : ============================================ How to install Softwares in Linux VM ============================================ Ex: git, maven, java, python etc... Package Managers in Linux => package managers are used to manage software packages (softwares) in linux machine. => Package means a software Ex: git, maven, java, python, httpd etc..... => package managers are specific to linux distribution Amazon Linux VM => yum Ubuntu Linux VM => apt # check git client installed or not git --version # install git client s/w sudo yum install git -y # check git installation path whereis git ======================================== How to change host name in linux vm ======================================== # set hostname sudo hostname # re-start session exit ============================================= How to find the location of files in linux? ============================================= => in linux we can use 'find' command to search file paths. # search for the files which are having name as f1.txt sudo find /home -name f1.txt # search for f2.txt file in entire linux machine sudo find / -name f2.txt # search for empty files inside /home sudo find /home -type f -empty # search for empty directories inside /home sudo find /home -type d -empty =============================================================== Assingment : host static website in linux vm (amazon linux) =============================================================== 1) whoami 2) pwd 3) date 4) cal 5) cd 6) ls 7) mkdir 8) rmdir 9) rm 10) touch 11) mv 12) cp 13) cat 14) tac 15) wc 16) head 17) tail 18) grep 19) vi 20) sed 21) zip 22) unzip 23) ping 24) wget 25) curl 26) ifconfig 27) ps aux 28) kill 29) useradd 30) passwd 31) userdel 32) usermod 33) groupadd 34) gpasswd 35) lid 36) id 37) groupmod 38) chmod 39) chown 40) yum install 41) whereis 42) uptime 43) free 44) top 45) find