join in WhatsApp channel for class notes : https://www.whatsapp.com/channel/0029Va9NnSdCHDyqwAoeIB1G ========= Linux OS ========= => Where we will use linux os in real-time => We will install devops tools in linux machines => Jenkis Server => Docker Server => K8S cluster => SonarQube server => Nexus Server => ELK stack => Grafana & Promethues servers => Ansible server => We will install database server software in linux machines => Real-Time Applications will be deployed in linux machines only ============= What is OS ? ============= => It is a software which acts as mediator between user and computer. => Users will communicate with computers using OS => Without OS we can't use any computer => OS provides platform to run our applications in computer. Ex: notepad, calculator, browser, tomcat.... => We have several operating systems in market Ex: Windows, Linux, Mac, Android, IOS etc... =========== Windows OS =========== => Developed by Microsoft company (Bill Gates) => Windows os is licensed (commercial) => Windows is single user based os => Security features are less in windows os (anti virus s/w required) => Windows is GUI based (graphical user interface) => Windows is recommended for personal use Ex: watch movies, play games, online classes.... ========== Linux OS ========== => Linux is community based os (not specific to any company) => Linux is free and open source os => Linux is Multi User based OS => Linux is highly secured (anti virus is not required) => Linux is CLI based os (command line interface) => Linux is highly recommended for business use (servers management) Ex: App Servers, DB servers, jenkins, docker, k8s, nexus.... ============== Linux History ============== => Linux OS developed by "Linus Torvalds" -> Linus Torvalds identified some challenges/issues in Unix OS -> Linus Torvalds identified one OS which is matching with his ideas i.e Minux os -> Linus Torvalds used Minux OS code and made some changes and released into market as new OS i.e Linux OS. (Li) nus + Mi (nux) = Linux ====================== Linux Distributions ====================== -> Linus Torvalds provided Linux OS source code for free of cost -> So many companies downloaded Linux OS source code and modified according to their requirement and released into market with their brand names Those are called as Linux Distributions/ Linux Flavours. -> We have 200+ Linux Distributions in the market. Ex: Amazon Linux, Ubuntu, Red HAT, suse, debian, kali, fedora.... ============================= How to setup Linux Machine ? ============================= Approach-1 : Download and Install Linux OS in our System. Approach-2 : Use Virtual Box and install Linux os as guest os Approach-3 : Setup Linux VM in AWS Cloud for free of cost ( t2.micro / t3.micro -> 1 year & monthly 750 hours you can use ) ================================================================================ *AWS Account Setup* : https://youtu.be/xi-JDeceLeI?si=4MgBX_H4_NPuzdT8 *Linux VM Setup in AWS* : https://www.youtube.com/watch?v=JMlQaTXvw5o *Connect Linux VM with MobaXterm* : https://youtu.be/uI2iDk8iTps?si=ZuZs0lQTxoRpbRMk *Connect Linux VM with putty* : https://youtu.be/GXc_bxmP0AA?si=HgSydrP89mPxv23s =========================== How to setup linux machine =========================== 1) Install Linux OS directley in our computer (physical machine) 2) Install Linux OS as Guest OS in windows machine using Virtual Box. 3) Take Linux Virtual Machine using Cloud Platform like AWS =================== Linux Architecture =================== 1) Shell : mediator between users and kernel - it will read commands executed by users - validate command syntax - convert command into kernel understandable format - shell will send instructions to kernel for execution 2) Kernel : heart of linux os (very imp) - It will recieve instructions from shell - kernel will convert command into hardware understandable format - It is mediator between shell and hardware 3) Hardware components : cpu, ram, io =================== Linux File System =================== => In Linux OS everything will be represented as File only. => Root Directory (/) is starting point of linux machine => Inside root directory we have several directories like below /home : It contains users home directories /home/ashok : Ashok home directory /home/raju : Raju home directory /home/ec2-user : ec2-user home directory /bin : Essentials ready-to-run-program (binaries) /boot : static files of boot loader /dev : device files /lib : Shared libraries /etc : host specific configurations /opt : Optional application software packages /tmp : Temporary files will be stored here /usr : user utilities and applications /mnt : Mounted File system /media : Removal Media ================= Linux Commands ================= whoami : Display logged-in username pwd : Display present working directory date : Display current date cal : Display current month calendar cal 2030 : Display 2030 calendar cd : change directory cd .. : go one step back from pwd cd : Go inside directory mkdir : Make directory (create folder) mkdir linux mkdir aws mkdir devops azure gcp rmdir : remove empty directory (delete) rmdir devops rm -rf : remove non-empty directory (recursively & forcefully) rm -rf devops ls : display present working directory content ls -l : long list the files in alphabetical order ls -lr : display files in reverse of alphabetical order ls -lt : display latest files on top ls -ltr : display old files on top ls -la : display hidden files touch : To create empty files in linux touch f1.txt touch f2.txt f3.txt rm : to delete file rm f1.txt rm -rf mv : For rename and move files & directories mv existing-name new-name mv presention-location new-location cat : create new file with data + append data to file + print file data # create new file with data cat > f1.txt # append data to existing file cat >> f1.txt # print file data from top to bottom cat f1.txt # print file data with line numbers cat -n f1.txt tac : To print file data from bottom to top tac f1.txt cp : copy data from one file to another file cp f1.txt f2.txt Note: If we want to copy data from multiple files then we should use cat command cat f1.txt f2.txt > f3.txt head : print first 10 lines of the file head f1.txt # print first 5 lines only head -n 5 f1.txt # print first 25 lines only head -n 25 f1.txt tail : print last 10 lines of the file tail f1.txt # print last 25 lines of data tail -n 25 f1.txt ### Note: To get latest log message from log file we will use tail command ### grep : grep stands for global regular expression print Note: Using grep command we can search for content in the file # print lines which contains java keyword grep 'java' fullstack.txt # ignore case-sensitive grep -i 'java' fullstack.txt # print along with line numbers grep -n 'java' fullstack.txt # print lines which doesn't contains 'java' keyword (invert match) grep -v 'java' fullstack.txt # tail with grep combination # search for java keyword in last 10 lines of file tail fullstack.txt | grep 'java' wc : word count # print no.of lines, no.of words, no.of letters in the file wc fullstack.txt ======================= Text Editors in Linux ======================= => vi (visual editor) it is default editor in linux machines. => Using 'vi' we can create new files and we can modify existing file data. => vi command is having 3 modes 1) command mode (just to open file) ex: vi f1.txt 2) insert mode (to edit the file) --------> press 'i' in keyword 3) esc mode (to come out from insert mode) --> press 'esc' in keyword # save the changes and close the file ====> :wq + enter # close the file without saving changes ===> :q! + enter Note: vi command will open the file if it is avilable otherwise it will create new file and it will open that file. ======================== file creation commands ======================= touch : to create empty file cat : create file with data (cat > f1.txt) cp : copy one file data into another file (cp f1.txt f2.txt) vi : create and open file for editing (vi f3.txt) ==================== reading file data ==================== cat : print file data from top to bottom tac : print file data from bottom to top head : print first 10 lines of file data tail : print last 10 lines of file data vi : open the file ============= SED Command ============= => SED stands for stream editor => SED is used to process the data (substitute, delete, print, insert) => Using SED command we can perform operations on the file without opening the file. => SED is very powerful command in linux => we will use below options with SED command s - substitute d - delete p - print i - insert # replace first occurance of linux with unix in every line sed 's/linux/unix/' os.txt # replace second occurance of linux with unix in every line sed 's/linux/unix/2' os.txt # replace third occurance of linux with unix in every line sed 's/linux/unix/3' os.txt # replace all occurances of linux keyword with unix sed 's/linux/unix/g' os.txt # substitute linux with unix and save changes in the file sed -i 's/linux/unix/g' os.txt # delete first line of the file sed -i '1d' os.txt # delete 4th line of the file sed -i '4d' os.txt # delete last line of the file sed -i '$d' os.txt # delete from 3rd line to last line sed -i '3, $d' os.txt # delete from 10th line to 15th line sed -i '10, 15d' os.txt # Add line at end of file sed -i '$a\i am from ashokit' os.txt # Add line at end of file sed -i '2i\i want to learn devops' os.txt # print 10th line only sed -n '10p' os.txt # print data from 2nd line to 5th line sed -n '2,5p' os.txt ================================= Working with Zip files in linux ================================= => Zip is used for files archieve (compress) ## syntax to create zip file : $ zip # create zip with all .txt files zip myfiles *.txt # extract zip file unzip myfiles.zip ====================== Networking commands ====================== ping : To check connectivity ping www.google.com ping www.facebook.com ping 192.168.1.34 wget : It is used to download files from internet wget curl : It is used to send http request to the url curl curl https://dummyjson.com/quotes/random ifconfig : To get IP address of our machine ============================================= How to find the location of files in linux? ============================================= => in linux we can use 'find' command to search file paths. # search for the files which are having name as f1.txt sudo find /home -name f1.txt # search for empty files inside /home sudo find /home -type f -empty # search for empty directories inside /home sudo find /home -type d -empty # print 30 days old files in linux machine sudo find /home -type f -mtime +30 ===================== process management ===================== Process management in Linux involves controlling and monitoring the execution of programs (processes) on the system. # display processing running with PID $ ps aux Note: Every process will have process id (PID) # kill process kill # terminate process immediatley (forcefully) kill -9 ================= User Management ================= => Linux is a multi user based OS => Multiple users can acces single linux machine and can perform multi tasking at time. => Within one linux machine we can create multiple user accounts. Note: "ec2-user" is a default user in "amazon linux" vm. ec2-user having sudo priviliges. Amazon Linux AMI ===> ec2-user Ubuntu AMI ==> ubuntu => when we create user account, for user one home directory will be created. ec2-user => /home/ec2-user john => /home/john smith => /home/smith # display all users created cat /etc/passwd # create user sudo useradd # set pwd for user sudo passwd # swith user account su # navigate to current user home directory cd ~ # delete user along with user home directory sudo userdel --remove # rename user sudo usermod -l /etc/passwd : This file contains user information /etc/shadow : This file contains user passwords in encrypted format =================================== Working with user groups in linux =================================== => When we create user in linux, for that user one user group also will be created with the given username. # display all groups cat /etc/group # create group in linux sudo groupadd # add user to group sudo usermod -aG # remove user from group sudo gpasswd -d # display users belongs to group sudo lid -g # check one user belongs to how many groups id # delete group sudo groupdel # changing group name $ sudo groupmod -n ======================================================================================= Assignment : create new user and connect with linux vm using newly created user account ======================================================================================= => To connect with Linux VM using our own user account , we need to enable Password Based Authentication in Linux VM. => To enable PasswordBasedAuthentication in linux vm then we need to modify below configuration files 1) sudoers 2) sshd_config ================================= What is sudoers file in Linux ================================= => It is very important configuration file in linux machine. => Using this file we can control which user can run command as a superuser. # print sudoers file content sudo cat /etc/sudoers # Open sudoers file sudo visudo # Add below line under root user ALL=(ALL) ALL => After making changes to close sudoers file => ( CTRL + X + Y + Enter) ======================================================== How to enable password based authentication for users ? ======================================================== => In linux vm, by default passwordauthentication is no => If we want to connect with linux vm by using username and password then we need to set passwordauthentication value as yes. => WE WLL MODIFY THIS IN "sshd_config" file. # Display sshd_configurration file data $ sudo cat /etc/ssh/sshd_config # Open file $ sudo vi /etc/ssh/sshd_config Note: Go to insert mode and enable pwdbasedauthentication as yes # restart sshd service # sudo systemctl restart sshd Note : Now we can connect with linux vm using username and pwd Step-1 : Open gitbash Step-2 : Execute below command $ ssh uname@public-ip ========================== File Permissions in Linux ========================== => Using file permissions we can secure our files and we can protect our file data. => We have 3 types of permissions in linux r => read w => write x => execute => file/directory permissions will be represented like below rwxrwxrwx f1.txt => file permissions contains 9 characters and those are divided into 3 parts first 3 characters => user/owner permissions middle 3 characters => group permissions last 3 characters => other users permissions => Lets under file permissions with diff scenarios Scenario-1 :: rw-r--r-- f1.txt user => read + write group => read others => read Scenario-2 :: rwxr-xr-x f1.txt user => read + write + execute group => read + execute others => read + execute Scenario-3 : rwx--xr-x f1.txt user => read + write + execute group => execute others => read + execute => To change file/directory permissions we will use 'chmod' command + => To add permission - => To remove permission # Giving execute permission for user $ chmod u+x f1.txt # giving write permission for group $ chmod g+w f1.txt # Remove execute permission for others $ chmod o-x f1.txt # give all permissions for group $ chmod g+rwx f1.txt # Remove all permissions for others $ chmod o-rwx f1.txt ==================================== File Permissions in Numeric Format ==================================== 0 => No Permission 1 => Execute 2 => Write 3 => ( 2 + 1) => Write + Execute 4 => Read 5 => (4 + 1) => Read + Execute 6 => (4 + 2) => Read + Write 7 => (6 + 1) => Read + Write + Execute # ugo + x chmod 111 f2.txt # ugo + w chmod 222 f2.txt # u+r g+rx o+rw chmod 456 f2.txt # u+rwx g+rw o+rx chmod 765 f2.txt # u-rwx g-rwx o+rwx chmod 7 f2.txt =============== chown command =============== => It is used to change file/directory ownership # change owner sudo chown new-owner # change owner-group sudo chown :new-group # change owner & group of file/directory sudo chown new-owner:new-group ========================== Package Managers in Linux ========================== => Package means a software Ex: git, maven, java, python etc..... => package managers are used to manage software packages (softwares) in linux machine. => package managers are specific to linux distribution Amazon Linux ===> yum Ubuntu Linux ===> apt # check git client installed or not git --version # install git client s/w sudo yum install git -y # check git installation path whereis git # check java installed or not java -version # install java s/w sudo yum install java -y # check java installation path whereis java ============================= Webserver Setup in Linux VM ============================ => Webserver is a software which is used to run websites / web applications. => Website means collection of web pages ex: login page, register page, dashboard page, about us page .... => Websites are divided into 2 types 1) static website 2) dynamic website => The website which gives same response for every user is called as static website. Ex: www.eenadu.net => The website which gives response based on logged in user is called as dynamic website. Ex : www.gmail.com => To run static websites we can use 'httpd' as webserver. => To run dynamic websites we can use 'tomcat, iis, jboss, weblogic as webservers. # install httpd webserver sudo yum install httpd -y # check httpd status sudo service httpd status # start httpd server sudo service httpd start Note: httpd webserver runs on http protocol with 80 port number. Note: To access our webserver we need to enable http protocol in Ec2 vm security group inbound rules. # Navigate to website content directory $ cd /var/www/html # create index.html file with website content $ sudo vi index.html => We can access our webserver using ec2-vm public ip. ================================ What is systemctl in linux ? ================================ => systemctl is used to manage services in linux machines. => using systemctl we can perform below operations a) start a service b) stop service c) re-start service # stop httpd server sudo systemctl stop httpd # start httpd server sudo systemctl start httpd # reload service sudo systemctl reload httpd ============================================= How to change hostname in vm (temporarly) ? ============================================= # set hostname $ sudo hostname # re-start session $ exit Note: connect back to vm then we can see configured hostname =================================== How to set hostname permanentley =================================== # update hostname in below file $ sudo vi /etc/hostname # restart the vm Note: After restart hostname configured in file will be reflected in terminal. ======================= 1) whoami 2) pwd 3) date 4) cal 5) cd 6) ls 7) mkdir 8) rmdir 9) rm 10) touch 11) mv 12) cp 13) cat 14) tac 15) wc 16) head 17) tail 18) grep 19) vi 20) sed 21) zip 22) unzip 23) ping 24) wget 25) curl 26) ifconfig 27) find 28) uptime 29) free 30) top 31) ps aux 32) kill 33) useradd 34) passwd 35) userdel 36) usermod 37) groupadd 38) groupdel 39) gpasswd 40) lid 41) id 42) groupmod 43) chmod 44) chown 45) yum install 46) yum remove 47) whereis 48) systemctl