Albara's blog

This blog for random codes that I use for solving minor problems or do some small tasks, most likely with no categories! I think it's easy for me to come here when I need to reuse something instead of rewriting it over and over again!

Clear the terminal history Raspberry Pi

history -c && history -w

Change Raspberry Pi password without having a password

Temporarily adding init=/bin/sh to the end of /boot/cmdline.txt will force the Rpi to boot to a root level console mode where the password for user pi can be changed.

console=serial0,115200 console=tty1 root=PARTUUID=d6b6a7aa-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-console init=/bin/sh

Reboot then use the following commands

mount -o remount, rw /
passwd pi 
sync
exec /sbin/init

Ref

NMAP (Network Mapper) on Raspberry Pi

Using a CentOS:

sudo yum install -y nmap

Using a Debian (Ubuntu/Raspbian):

sudo apt-get install -y nmap

Using the MAC Address To Identify a Pi

sudo nmap -sP 192.168.0.0/24

The default SSH user and password on Raspberry Pi

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory:

username: pi
password: raspberry

To start SSH service

sudo service ssh start

To check the status of SSH service

sudo service ssh status

Copying file to multiple folders

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory:

for dir in *; do [ -d "$dir" ] && cp -f /var/www/html/index.php "$dir" ; done

Zipping folder in Linux

har is the target folder, har.zip is the zipped file:

zip -r har.zip har

Copying file to multiple folders

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory:

for dir in *; do [ -d "$dir" ] && cp -f /var/www/html/index.php "$dir" ; done

Changing the EC2 size

follow the direction here: url: ref (click here)
then use SSH to check the size

df -h
lsblk

Changing the EC2 type

AWS changing the type :ref (click here)
AWS pricing :ref (click here)
AWS types (CPU and RAM) : ref (click here)
then use SSH to check the size

SSH to AWS EC2

ssh -i aaa.pem ec2-user@1.2.3.4

Changing the owner of the file to apache on EC2 AWS

With this command, Apache server (PHP files) will be able to do changes in the files/directories in the server

sudo chown -R apache:apache ./file.json

With this command, FileZilla (FTP app) will be able to do changes in the files/directories in the server

sudo chown -R ec2-user:apache ./file.json

Set Different Volume Levels For Each App on Mac pro

Here is an app url: click here

Download and run Tensorflow on EC2 AWS

With these commands, you'll be able to install tensorflow in a virtual environment and run it:

#---------------------------------------------
#Create a virtual environment:
#---------------------------------------------
python3 -m venv --system-site-packages ./venv
source ./venv/bin/activate  # sh, bash, or zsh
pip install --upgrade pip
#---------------------------------------------
#Install TensorFlow
#---------------------------------------------
pip install --upgrade tensorflow
deactivate
#---------------------------------------------
#run python file out of the venv
#---------------------------------------------
./venv/bin/python file.py

Get MAC address of rasperrypi for both eth and wlan

With these commands, you'll be able to get the MAC address of eth0 and wlan0 and print them in whatever text format you wish:

IFACE_E=eth0
read MAC_E </sys/class/net/$IFACE_E/address
IFACE_W=wlan0
read MAC_W </sys/class/net/$IFACE_W/address
MACX=${IFACE_E}${MAC_E}${IFACE_W}${MAC_W}
MACX2=$(echo $MACX | sed 's/[:"]//g')
echo $MACX2

Camera does not work on macbook-Pro

Run the following command

sudo killall VDCAssistant

Raspberry Pi: to find the version e.g, 3 or 4

Run the following command

cat /proc/device-tree/model

Raspberry Pi: SD Card to img on macbook

Find the disk number

diskutil list

run the command with the disk number (for example here is the disk2):

sudo dd if=/dev/rdisk2 of=aaa.img bs=1m

Raspberry Pi: img to SD Card on macbook

Find the disk number

diskutil list

Unmount the disk:

diskutil unmountDisk /dev/disk2

run the command with the disk number (for example here is the disk2):

sudo dd bs=1m if=aaa.img of=/dev/rdisk2

Eject the disk. To safely remove the SD card:

diskutil eject /dev/rdisk2

PHP: To read the csv file without having to deal with new line issue

Here is the code, the csv file will be save in array: $csv

$filename="aaa.csv";
$file = fopen($filename,"r");
while(! feof($file))
  {
  $csv[]=fgetcsv($file);
  }
fclose($file);

Apahce: Increase the HTTP request size (GET).

First, open the file:

sudo vim /etc/httpd/conf/httpd.conf

Add the limitationm line (by bytes)

LimitRequestBody 10000000000
LimitRequestLine 10000000000
LimitRequestFieldSize 10000000000

Then restart the server

sudo service httpd restart

PHP: Increase the HTTP request size (POST).

First, open the file:

sudo vim /etc/php.ini

Add the limitation size, or put 0 if you dont want to have any limitation.

post_max_size = 0

PHP: file_get_contents doesn't work anymore on AWS EC2.

You've to add ssl verify array:

$arrContextOptions=array("ssl"=>array("verify_peer"=>false,"verify_peer_name"=>false,),);

Then you could call the file_get_contents

$output = file_get_contents('http://website.com', false, stream_context_create());

Shell: run in the background without saving the log file: nohup.out:

Note first add the follwing line in the sh file:

#!/bin/bash

Then give the following permsion te the sh file:

sudo chmod +x aaa.sh

ex. if aaa.sh was your sh file you want to run then the format will be:

sudo nohup sh aaa.sh >/dev/null 2>&1 &

Shell: Get the size of folder in a path:

ex. if you want to see the size of folders in the home directory:

sudo du -h --max-depth=1 /home/

Shell: Infinite loop to kill a process by name:

ex. tee is a keyword for a command (replace it with any other word)

while true ; do sudo pkill -f tee ; done

Pyhton: Create empty array

Here is the source code:

import numpy as np
np.array([])

EC2: Create swap area

Ref: click here If swapfile already exist

sudo swapoff -a

The swap file is 4 GB (128 MB x 32):

sudo dd if=/dev/zero of=/swapfile bs=128M count=32

Update the read and write permissions

sudo chmod 600 /swapfile

Set up the swap area:

sudo mkswap /swapfile

Make the swap file available

sudo swapon /swapfile

Verify

sudo swapon -s

Start the swap file at boot

sudo vi /etc/fstab

Add the following line at the end

/swapfile swap swap defaults 0 0

Linux: Run script in the background (even after exiting the terminal)

Using nohup command you can run a shell script file as follows:

nohup sh ./cut-all.sh &

Linux: print a size of directory

If the target directory is folder, here is the command:

sudo du -sh ./folderA

Python3: Virtual environment

Create:

python3 -m venv ./venv

Activate:

source ./venv/bin/activate

Python: check if a URL (file, image,page, etc. exist or not without loading the whole file)

import requests

def is_there(path):
    r = requests.head(path)
    return r.status_code == requests.codes.ok

Python/pandas: Mode with a "group by" in pandas

Let's assume you have a pandas data frame df and you want to group by column A and calculate the mode of column B then calculate the mean of B afterward.

import numpy as np
def mode(a):
    u, c = np.unique(a, return_counts=True)
    return u[c.argmax()]
#####
g=pd.DataFrame(df[['A','B']].groupby(['A']).apply(mode) )
g.columns=["B"]
XMODE=g['B'].mean()

Ref: click here

Python/pandas/matplotlib: Fill the area between 2 pandas curves

Let's assume you have a pandas data frame df and you want to fill the area between columns A and B. With a grey color and transparency of 0.5.

plt.fill_between(df.index, df['A'],df['B'], interpolate=True, color='grey', alpha=0.5)

Ref: click here

Linux: copy directory dir1 to another one dir2 without losing the contents in the target directory

rsync -r dir1/ dir2

Python/Pandas: filter a columns of data-frame based on contains a partial of string

Let's assume we have a data-frame A and a column called 'user'. Let's say we want to filter the users that contain 1234

B=A[A['user'].str.contains("1234")]

If we want the once that doesn't contain 1234

B=A[~A['user'].str.contains("1234")]

Linux: go to the end of the command in htop

To go to the beginning

 Ctrl+A

To go the end

 Ctrl+E

Linux: protect folder by cookies

Edit the following file:

sudo vim /etc/httpd/conf/httpd.conf

Add the following commandL


RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_COOKIE} !user=admin [NC]
RewriteRule ^ https://example.net/error.php [NC,L]

Then restart the apache server

sudo /etc/init.d/httpd restart

Python/panda: read csv using read_csv via https url

Use the follwoing commandas:

import io
import requests
path="https://example.com/a.csv"
s=requests.get(path).content
cont=io.StringIO(s.decode('utf-8'))
dfx=pd.read_csv(cont,header=None)

AWS/EC2: crontab

To edit the root crontab

sudo vim /etc/crontab

AWS/EC2: certbot & Let's Encrypt certificates

Follow the instruction in the following link: click here

sudo yum update
sudo yum install python3 python3-venv libaugeas0
sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip
sudo /opt/certbot/bin/pip install certbot certbot-apache
sudo ln -s /opt/certbot/bin/certbot /usr/bin/certbot

Certbot edit the apache configuration automatically:

sudo certbot --apache

Or, just get a certificate:

sudo certbot certonly --apache

Auto renew:

echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

AWS/EC2: set up an EC2/Apache/PHP/MySQL

Step 1 (set up an EC2 instance)

Go to: https://aws.amazon.com/
Create an account (you could use your Amazon account)
Create an EC2 instance.

Step 2 (access the EC2 instance via SSH)

(1) From the Command line we have to change the permission of the Key to make it private:

chmod 400 ttest.pem

(2) Then we will need to use the following info:

Username: ec2-user

Public DNS: *.*.amazonaws.com        [or you could use the IP instead]

Key: test.pem

(3) From the command line make the SSH connection

ssh -i ttest.pem ec2-user@*.*.amazonaws.com

Step 3 (set up Apache/PHP/MySql)

(0) you may update the packages

sudo yum update –y
#sudo yum update

(1) download the PHP package

#sudo yum install -y httpd24 php56 mysql55-server php56-mysqlnd
sudo yum install -y httpd24 php70 mysql56-server php70-mysqlnd

(2) Start the HTTP service

sudo service httpd start
#sudo chkconfig httpd on
#chkconfig --list httpd

(3) Go the folder of www (online)

cd /var/www/

(4) give permission to add/delete and edit for the ec2-user

sudo chown -R ec2-user:apache /var/www

AWS/EC2: Step 4 (setup mysql_secure_installation)

(1) Start MySQL server.

sudo service mysqld start

(2) Run mysql_secure_installation.

sudo mysql_secure_installation

(3) Setup password for example

username: root

Password: 123456

(4) In case you want to stop MySql

sudo service mysqld stop

(5) Auto run when setup server

sudo chkconfig mysqld on

AWS/EC2: Step 5 (Install phpMyAdmin)

sudo yum install php70-mbstring.x86_64 php70-zip.x86_64 -y
sudo service httpd restart
cd /var/www/html
wget https://www.phpmyadmin.net/downloads/phpMyAdmin-latest-all-languages.tar.gz
mkdir phpMyAdmin && tar -xvzf phpMyAdmin-latest-all-languages.tar.gz -C phpMyAdmin --strip-components 1
rm phpMyAdmin-latest-all-languages.tar.gz
sudo service mysqld start

To access mysql:

mysql -u root -p

AWS/EC2: cinstall ffmpeg

Follow the instruction:

sudo cd /usr/local/bin
sudo mkdir ffmpeg2
sudo cd ffmpeg2
sudo wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz # https://johnvansickle.com/ffmpeg/
sudo tar xvf ffmpeg-git-amd64-static.tar.xz
sudo mv ffmpeg-git-20220722-amd64-static/ ./ffmpeg
sudo cd ..
sudo rm -r ffmpeg2
sudo cd  ffmpeg
./ffmpeg -version
sudo ln -s /usr/local/bin/ffmpeg/ffmpeg /usr/bin/ffmpeg

Then you could just use the following command from anywhere: Auto renew:

sudo ffmpeg -i vvv.mov vvv.mp4

Ref: 1 and 2

AWS/EC2: remove http logs

sudo rm -r /var/log/httpd/

RPI: change resultion

Here is a configuratio as an example

xrandr --output HDMI-1 --mode 1920x1080

RPI: change orientaion and resultion

Here is a configuratio as an example

nano ~/.profile
# to find HDMI-1 or other run: xrandr
# then run the command with the resution you want: cvt 1920 1080 60
sudo xrandr --newmode "1920x1080_60.00"  173.00  1920 2048 2248 2576  1080 1083 1088 1120 -hsync +vsync
sudo xrandr --addmode HDMI-1 "1920x1080_60.00"
sudo xrandr --output HDMI-1 --rotate right

Ref1 Ref2 Ref3

RPI: trun the screen on or off

On (and rotate it right)

xrandr --output HDMI-1 --auto --rotate right

Off

xrandr --output HDMI-1 --off

Ref

Rasberry PI/: Rotate LCD touch screen 7in 90degree to the right

Rotate display

sudo nano /boot/config.txt
display_lcd_rotate=2

Rotate touch

# create sh file (e.g screen.sh) and put the command in it
xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 0 1 0 -1 0 1 0 0 1

Open autostart and run the file from it

sudo nano /etc/xdg/lxsession/LXDE-pi/autostart
sudo screen.sh

Here is the rest angels:

Rotate left 90 = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 0 -1 1 1 0 0 0 0 1
Rotate 180 = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' -1 0 1 0 -1 1 0 0 1
Rotate right 90 = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 0 1 0 -1 0 1 0 0 1
Return to upright = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 1 0 0 0 1 0 0 0 1

Ref1 Ref2

AWS/Apache: check the url requests

Open config file

sudo vim /etc/httpd/conf/httpd.conf

Put the following, and change example.com to your domain


    SetHandler server-status
    Require host example.net
    AllowOverride None
    # Allow open access:
    Require all granted

Then restart apache

sudo service httpd restart

Rpi4: black monitor issue

When the moniotr HDMI does not display anything except black eventhoug the boot works fine.
Open config file

sudo nano /boot/conifg.txt

Uncomment the followig line

hdmi_safe=1

Then restart the device

reboot

Linx: find file by type in subfolders

When the moniotr HDMI does not display anything except black eventhoug the boot works fine.
Open config file

find . -name "*.csv"

Ref: click here

Linx: delete file by type in subfolders

When the moniotr HDMI does not display anything except black eventhoug the boot works fine.
Open config file

find . -name "*.csv" -type f -delete

Ref: click here