$ ssh [email protected] ssh [email protected] ssh [email protected] -weight: 600;">sudo -weight: 500;">apt -weight: 500;">update # Check for updates -weight: 600;">sudo -weight: 500;">apt -weight: 500;">upgrade # Install updates pwd # See your current directory ls # List files and folders -weight: 600;">sudo -weight: 500;">apt -weight: 500;">update # Check for updates -weight: 600;">sudo -weight: 500;">apt -weight: 500;">upgrade # Install updates pwd # See your current directory ls # List files and folders -weight: 600;">sudo -weight: 500;">apt -weight: 500;">update # Check for updates -weight: 600;">sudo -weight: 500;">apt -weight: 500;">upgrade # Install updates pwd # See your current directory ls # List files and folders -weight: 600;">sudo useradd -m grace # Create user with home folder -weight: 600;">sudo passwd grace # Set password logout # Log out from root ssh [email protected] -weight: 600;">sudo useradd -m grace # Create user with home folder -weight: 600;">sudo passwd grace # Set password logout # Log out from root ssh [email protected] -weight: 600;">sudo useradd -m grace # Create user with home folder -weight: 600;">sudo passwd grace # Set password logout # Log out from root ssh [email protected] mkdir Project # Create a folder cd Project # Enter the folder touch main.py # Create a Python file mkdir data # Create a sub-folder ls # Verify folder and files mkdir Project # Create a folder cd Project # Enter the folder touch main.py # Create a Python file mkdir data # Create a sub-folder ls # Verify folder and files mkdir Project # Create a folder cd Project # Enter the folder touch main.py # Create a Python file mkdir data # Create a sub-folder ls # Verify folder and files nano main.py nano main.py nano main.py cat main.py less main.py more main.py cat main.py less main.py more main.py cat main.py less main.py more main.py -weight: 500;">wget https://example.com/data.csv # Download a file tar -xzf archive.tgz # Extract compressed files -weight: 500;">wget https://example.com/data.csv # Download a file tar -xzf archive.tgz # Extract compressed files -weight: 500;">wget https://example.com/data.csv # Download a file tar -xzf archive.tgz # Extract compressed files scp main.py [email protected]:/home/grace/from_local/ scp main.py [email protected]:/home/grace/from_local/ scp main.py [email protected]:/home/grace/from_local/ cd from_local python3 main.py cd from_local python3 main.py cd from_local python3 main.py - Setting up and managing servers: Configuring the machines where your data tools run. - Scheduling jobs: Using CRON to trigger data pipelines automatically. - Debugging failures: Connecting via SSH to investigate logs when a pipeline breaks. - Moving and managing files: Handling raw data before it lands in storage like S3. - Installing tools: Setting up Python, Spark, Airflow, and other software on a server. - Monitoring resources: Checking server memory, disk usage, and overall health. - Connect to a virtual Linux Server - Manage files on the server - The server's IP address - My username - Type yes to accept the server key. - Enter your password (it won't show). - Press Enter, and you're in! - Paste your text or code - Press Ctrl + O to save - Press Ctrl + X to exit - Every tool in a data engineering pipeline runs on a Linux server to navigate, organize, and run tasks. - SSH is your bridge between your PC and the server. - Always -weight: 500;">update your server and create a personal user before anything else. - Start small: create folders, files, and scripts, then automate tasks. - Everything you do here mirrors real world data engineering work, like managing pipelines, logs, or datasets.