...
Create and update the new VM
Prepare environment for creating new zones
Prepare script to get dependencies for components
Create a zone for local repositoryCreate
Add packages and source files to the local repository in this zone
Create the new zone for components building
Build components
1. Create and update the new VM
...
Code Block |
---|
|
root@dilos:~# zfs create -o mountpoint=/zones rpool/zones
root@dilos:~# mkdir /export/builds |
2. Add the admin userUpdate the /etc/apt/sources.list
file:
Code Block |
---|
|
root@dilos:~# useradd -m -g staff admin
root@dilos:~# passwd admin
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
root@dilos:~# echo cat > /etc/apt/sources.list <<EOT
deb http://10.20.50.188/dilos4 du4-unstable main contrib non-free
deb http://10.20.50.188/dilos dg2-unstable main contrib non-free
deb-src http://10.20.50.188/dilos4 du4-unstable main contrib non-free
EOT
apt-get update |
2. Add the admin user:
Code Block |
---|
|
root@dilos:~# useradd -m -g staff admin
root@dilos:~# passwd admin
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
root@dilos:~# echo "admin ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers.d/admin |
...
Code Block |
---|
|
cat > create_zone.sh <<SCRIPT
#!/bin/bash
TYPE_ZONE=\$1
IP=\$2
SER=\$3
cat > \${HOME}/\${TYPE_ZONE}.cmd <<EOT
create
set zonename=\${TYPE_ZONE}
set zonepath=/zones/\${TYPE_ZONE}
set autoboot=false
set ip-type=exclusive
add net
set physical=iz\${SER}
set global-nic=vsw0
add property (name=ip,value="\${IP}.\${SER}")
add property (name=gateway,value="\${IP}.254")
add property (name=netmask,value="255.255.255.0")
add property (name=primary,value="true")
end
add attr
set name=resolvers
set type=string
set value=8.8.8.8
end
add fs
set dir="/export/home/admin"
set special="/export/home/admin"
set type="lofs"
end
add fs
set dir="/export/builds"
set special="/export/builds"
set type="lofs"
end
add fs
set dir="/data/repo"
set special="/data/repo"
set type="lofs"
end
verify
commit
exit
EOT
zonecfg -z \${TYPE_ZONE} -f \${HOME}/\${TYPE_ZONE}.cmd zoneadm -z \2>&1 | /usr/bin/tee -a /var/tmp/${TYPE_ZONE} install &&-bts.log
zoneadm -z \${TYPE_ZONE} install2> &1 boot
echo "127.0.0.1 localhost.\| /usr/bin/tee -a /var/tmp/${TYPE_ZONE} \-bts.log
zoneadm -z ${TYPE_ZONE}" >> boot 2>&1 | /zones/\usr/bin/tee -a /var/tmp/${TYPE_ZONE}-bts.log
echo "127.0.0.1 localhost.${TYPE_ZONE} ${TYPE_ZONE}" >> /zones/${TYPE_ZONE}/root/etc/hosts
SCRIPT
|
Make it executable:
Code Block |
---|
|
chmod 777 ./create_zone.sh |
Info |
---|
Now you will be able to run it with the format of ./create_zone.sh <zone-name> <zone-ip-domain> <zone-ip-node> , for example!: ./create_zone.sh zone02 192.168.0 3
|
3. Prepare script to get dependencies for components.
Create a folder for the local repository and enter it:
Code Block |
---|
|
mkdir -p /data/repo/pkg
cd /data/repo/pkg
|
res=$(grep -e 'WARNING:' -e 'warning:' -e 'W:' -e 'error:' -e 'E:' -e 'cannot' /var/tmp/${TYPE_ZONE}-bts.log | sort -u)
if [[ "$res" == "" ]]; then
echo " =========== Zone ${TYPE_ZONE} is successfully created and ready to use ============="
else
echo " =========== Zone ${TYPE_ZONE} creation log ============="
echo "$res"
echo "$res" | sed '/W: --force-yes is deprecated/d; /\/man\//d'
if [[ "$res" != "" ]]; then
echo " ========================================================"
echo "Please stop and report about zone bootstrap errors!"
fi
fi
SCRIPT
|
Make it executable:
Code Block |
---|
|
chmod 777 ./create_zone.sh |
Info |
---|
Now you will be able to run it with the format of ./create_zone.sh <zone-name> <zone-ip-domain> <zone-ip-node> , for example!: ./create_zone.sh zone02 192.168.0 3
|
3. Prepare script to get dependencies for components.
Create a folder for the local repository and enter it:
Code Block |
---|
|
mkdir -p /data/repo/pkg
cd /data/repo/pkg
|
To make the following script workable you have to create the list_component
file with a content from here.
...
Code Block |
---|
|
cat > make_repo.sh <<EOT
#!/bin/bash
#DEP_FILE=""
DEP_FILE="req.txt"
LOG_FILE=""
#LOG_FILE="/data/repo/repo.log"
function print_log {
if [[ "\$LOG_FILE" != "" ]]; then
echo "\$1" >> \$LOG_FILE
fi
}
## Loop for get build components dependencies
echo "" > list_to_check
echo "" > final_list
sed -i '/^[ \\t]*\$/d' list_component
echo "" >> list_component
echo_component
echo "" > temp
if [[ "\$DEP_FILE" != "" >> list_component]]; then
echo "" > temp\$DEP_FILE
fi
while read comp; do
## Skip empty line
if [[ "" == "\$comp" ]]; then
continue
fi
echo -e "\\033[33m =====> Getting depends for \$comp... \\033[0m"
res=\$(apt-rdepends --build-depends --follow=DEPENDS \$comp)
echo "\$res" | grep 'Build-Depends:' > tmp
echo "\$res" | grep 'Build-Depends-Indep:' >> tmp
sed 's/^[ \\t]*Build-Depends:[ \\t]*//; s/^[ \\t]*Build-Depends-Indep:[ \\t]*//' tmp | sed 's/[ \\t]*(.*)[ \\t]*\$//' >> temp
rm -f tmp
if [[ "\$DEP_FILE" != "" ]]; then
sed "s/^/\$comp:/g" temp >> \$DEP_FILE
fi
done < list_component
## Preparation of initial list of dependencies
sort -u temp > list_to_check
sed -i '/^[ \\t]*\$/d' list_to_check
end=\$(cat list_to_check)
echo "" >> list_to_check
mkdir /data/repo/pkg/_tdir
apt-get -R /data/repo/pkg/_tdir update
## Loop for get packages dependencies
while [[ "\$end" != "" ]]; do
echo "" > temp
while read comp; do
## Skip empty line
if [[ "" == "\$comp" ]]; then
continue
fi
## Skip packages that are already checked for dependencies
res=\$(grep "^[ \\t]*\$comp[ \\t]*\\\$" final_list)
if [[ "" != "\$res" ]]; then
continue
fi
echo "=====> Getting depends for \$comp..."
apt-get -R /data/repo/pkg/_tdir install --dry-run \$comp | grep "^Inst " | sed 's/^Inst //; s/ .*\$//' > tmp
cat tmp >> temp
## Add checked package to the list of already checked packages
echo "\$comp" >> final_list
if [[ "\$DEP_FILE" != "" ]]; then
sed "s/^/\$comp:/g" temptmp >> \$DEP_FILE
fi
done < list_to_check
## Remove repeated records from the list
sort -u temp > list_to_check
## Remove empty lines if any
sed -i '/^[ \\t]*\$/d' list_to_check
sed -i '/^[ \\t]*\$/d' final_list
end=\$(cat list_to_check)
echo "" >> list_to_check
done
## Loop for check and download dependencies
rmdir --ignore-fail-on-non-empty /data/repo/pkg/_tdir
sed -i '/^[ \\t]*\$/d' final_list
sort -u final_list > temp
mv -f temp final_list
echo "" >> final_list
rm -f list_to_check tmp
while read pkg; do
if [[ "" == "\$pkg" ]]; then
continue
fi
echo -e "\\033[36m =====> Check \$pkg... \\033[0m"
## Check if a package exists
res=\$(apt-cache show \$pkg | grep "Package: \$pkg")
if [[ "\$res" != "" ]]; then
## Download existed package
echo -e "\\033[35m =====> Downloading package \$pkg... \\033[0m"
res=\$(apt-get -R /data/repo/pkg/_tdir download \$pkg | grep 'E:' | sed -n 's/E://')
if [[ "\$res" == "" ]]; then
print_log "\$pkg : OK"
else
echo -e "\\033[31;1;4m \$res \\033[0m"
print_log "\$pkg: \$res"
if [[ "\$DEP_FILE" != "" ]]; then
echo "=======================================" >> errors.log
echo "\$pkg: \$res" >> errors.log
grep ":\$pkg" \$DEP_FILE >> errors.log
fi
fi
else
## Put error message if package doesn't exist
echo -e "\\033[31;1;4m\$pkg: No package found \\033[0m"
print_log "\$pkg: No package found"
if [[ "\$DEP_FILE" != "" ]]; then
echo "=======================================" >> errors.log
echo "\$pkg: No package found" >> errors.log
grep ":\$pkg" \$DEP_FILE >> errors.log
fi
fi
done < final_list
## Downloading *.deb packages
echo -e "\\033[32;1m =====> Uploading *.deb packages... \\033[0m"
reprepro -b /data/repo --export=never -C main includedeb du4-prebuild /data/repo/pkg/*.deb
## Loop for download sources
while read component; do
if [[ "" == "\$component" ]]; then
continue
fi
echo -e "\\033[33;1m =====> Downloading sources for \$component... \\033[0m"
apt-get source --download-only \$component
echo -e "\\033[32;1m =====> Uploading sources for \$component... \\033[0m"
reprepro -b /data/repo --export=never -C main includedsc du4-prebuild \$component*.dsc
rm -rf \$component*
done < list_component
## Creating index for local repo
echo -e "\\033[32;1m =====> Creating indices for local repo... \\033[0m"
reprepro -Vb /data/repo export du4-prebuild
echo -e "\\033[32;1m =====> DONE \\033[0m"0m"
rm -fr /data/repo/pkg/_tdir
if [[ "\$LOG_FILE" != "" ]]; then
echo "Downloaded package list is put to \$LOG_FILE"
fi
EOT
|
Make this file executable:
Code Block |
---|
|
chmod 777 ./make_repo.sh |
If you have to get the error log then put the log name to the string LOG_FILE=""
, for example:
Code Block |
---|
LOG_FILE="/export/admin/repo.log" |
4. Create a zone for local rpository
...
Make this file executable:
Code Block |
---|
|
cdchmod ~777 ./createmake_zonerepo.sh zonerepo 192.168.0 2
zlogin zonerepo
|
Info |
---|
All the following commands are executed in the zonerepo. |
Update the /etc/apt/sources.list
file:
Code Block |
---|
|
cat > /etc/apt/sources.list <<EOT
deb http://10.20.50.188/dilos4 du4-unstable main contrib non-free
deb http://10.20.50.188/dilos dg2-unstable main contrib non-free
deb-src http://10.20.50.188/dilos4 du4-unstable main contrib non-free
EOT
apt-get update
|
If you have to get the error log then put the log name to the string LOG_FILE=""
, for example:
Code Block |
---|
LOG_FILE="/export/home/admin/repo.log" |
4. Create a zone for the local repository
Create the new zone and enter it:
Code Block |
---|
|
cd ~
./create_zone.sh zonerepo 192.168.0 2
zlogin zonerepo
|
Info |
---|
All the following commands are executed in the zonerepo. |
Install packets required for working in the zone:
...
Code Block |
---|
|
echo -n "Codename: du4-prebuild
Suite: prebuild
Version: 4.0
Origin: DilOS
Label: DilOS prebuild
Description: DilOS Prebuild Repository
Architectures: solaris-i386 solaris-sparc source
Components: main contrib non-free
##SignWith: default
DebIndices: Packages Release . .gz .bz2 .xz
DscIndices: Sources Release . .gz .bz2 .xz
Contents: . .gz .bz2 .xz
" > /data/repo/conf/distributions
|
5. Add packages and source files to the local repository:
Code Block |
---|
|
cd /data/repo/pkg
./make_repo.sh
exit
|
Creating of the local repository can take a long time. So, you can run it with the screen utility.
Expand |
---|
title | Running with the screen utility |
---|
|
If you are connected by SSH and would like to turn off SSH session or even temporary turn off your computer, then you have enter the following command for tests running: Code Block |
---|
screen ./make_repo.sh |
Then you can in any time press Ctrl-A d and after that close your SSH session. When you would like to get back, login again to SSH please and enter the command: |
6. Create the new zone for components building
Create the new zone and enter it:
Code Block |
---|
|
./create_zone.sh zonebld 192.168.0 3
zlogin zonebld
|
Info |
---|
All the following commands are executed in the zonebld. |
Prepare the zone for components building. To do this please copy this script to, let say, the prepare.sh
file or execute the following command:
Code Block |
---|
|
cat > prepare.sh <<EOT
#!/bin/bash
WORKSPACE=/export/home/du4
apt-get clean
apt-get autoclean
apt-get remove libiconv-dev -y
apt-get install -y acl iputils-ping kbd ksh scheduler system-data-console-fonts git
apt-get install -y traceroute gmake build-essential man-db po-debconf po4a debhelper
apt-get install -y libx11-6 libxi6 libxtst6 liblua5.2-dev libnghttp2-dev libjansson-dev
apt-get install -y mawk doxygen graphviz libdb-dev libgdbm-dev libsqlite3-dev libpq-devCodename: du4-prebuild
Suite: prebuild
Version: 4.0
Origin: DilOS
Label: DilOS prebuild
Description: DilOS Prebuild Repository
Architectures: solaris-i386 solaris-sparc source
Components: main contrib non-free
##SignWith: default
DebIndices: Packages Release . .gz .bz2 .xz
DscIndices: Sources Release . .gz .bz2 .xz
Contents: . .gz .bz2 .xz
" > /data/repo/conf/distributions
|
5. Add packages and source files to the local repository
Code Block |
---|
|
cd /data/repo/pkg
./make_repo.sh
exit
|
Creating of the local repository can take a long time. So, you can run it with the screen utility.
Expand |
---|
title | Running with the screen utility |
---|
|
If you are connected by SSH and would like to turn off SSH session or even temporary turn off your computer, then you have enter the following command for tests running: Code Block |
---|
screen ./make_repo.sh |
Then you can in any time press Ctrl-A d and after that close your SSH session. When you would like to get back, login again to SSH please and enter the command: |
6. Create the new zone for components building
Create the new zone and enter it:
Code Block |
---|
|
./create_zone.sh zonebld 192.168.0 3
zlogin zonebld
|
Info |
---|
All the following commands are executed in the zonebld. |
Prepare the zone for components building. To do this please copy this script to, let say, the prepare.sh
file or execute the following command:
Code Block |
---|
|
cat > prepare.sh <<EOT
#!/bin/bash
WORKSPACE=/export/home/du4
apt-get clean
apt-get autoclean
apt-get remove libiconv-dev -y
apt-get install -y bison dhbuild-execessential cmake docbook-xml docbook-xsl libgtestgit dpkg-dev screen gmake
cp /etc/apt/sources.list /etc/apt/sources.list.bak
echo "" > /etc/apt/sources.list
echo "deb [trusted=yes] http://192.168.0.2/dilos du4-prebuild main contrib non-free" >> /etc/apt/sources.list
echo "deb-src [trusted=yes] http://192.168.0.2/dilos du4-prebuild main contrib non-free" 2>/dev/null >> /etc/apt/sources.list
cp /etc/apt/sources.list /etc/apt/sources.list.dev
apt-get update -y
#test -f /var/lib/dpkg/lock && rm -f /var/lib/dpkg/lock
#test -f /var/cache/apt/archives/lock && rm -f /var/cache/apt/archives/lock
#dpkg --configure -a
rm -rf \${WORKSPACE} && mkdir -p \${WORKSPACE}
apt-get upgrade -y
env DEBIAN_FRONTEND=noninteractive apt-get install -y -f
env DEBIAN_FRONTEND=noninteractive apt-get install -y openssh-server
EOT
|
...
Code Block |
---|
|
cat > build_all.sh <<EOT
#!/bin/bash
BLD_PARALLEL="4"
WORKSPACE=/export/home/du4
#STOP_ON_ERROR=1
STOP_ON_ERROR=0
#GCC_COMPILER="gcc-10"
GCC_COMPILER=""
# Build a component: \$1 - component name, \$2 - stop-on-error flag
function build_component {
local START_TIME
local END_TIME
local BLD_RESULT
BLD_RESULT="/data/repo/du/\$1"
test -z "\$1" && exit 1
test -d \${BLD_RESULT} && rm -rf \${BLD_RESULT}
DEB_BUILD_OPTIONS="nocheck notest parallel=\${BLD_PARALLEL}"
export DEB_BUILD_OPTIONS
test -d \${WORKSPACE}/\$1 && rm -rf \${WORKSPACE}/\$1
mkdir -p \${WORKSPACE}/\$1
cd \${WORKSPACE}/\$1 \${WORKSPACE}/\$1
START_TIME=\$(date +%s)
echo "====================== \$1 ======================" | tee berror.log
if [[ \$2 -eq 0 ]]; then
apt-get build-dep -y \$1 >> berror.log || return 0 2>&1
apt-get source --compile \$1 >> berror.log || return 0 2>&1
else
apt-get build-dep -y \$1 >> berror.log || return 1 2>&1
apt-get source --compile \$1 >> berror.log || return 1 2>&1
fi
fiEND_TIME=\$(date +%s)
printf "%06d %s\\n" \$((\${END_TIME} - \${START_TIME})) \$1 >> \${WORKSPACE}/tmpbt.log
mkdir -p \${BLD_RESULT}
cp -ax *.deb \${BLD_RESULT}/ || exit 1
cp -ax *.changes \${BLD_RESULT}/ || exit 1
cp -ax *.buildinfo \${BLD_RESULT}/ || exit 1
}
if [[ "\$GCC_COMPILER" != "" ]]; then
echo "====================== Building GCC ======================"
build_component \$GCC_COMPILER 1
if [[ \$? -eq 1 ]]; then
exit 1
fi
dpkg -i \$WORKSPACE/\$GCC_COMPILER/*.deb
fi
if [[ ! -f \$WORKSPACE/build_components ]]; then
cp /data/repo/pkg/list_component \$WORKSPACE/build_components
fi
rm -f tmpbt.log
while read COMP; do
if [[ "" == "\$COMP" ]] || [[ "\$GCC_COMPILER" == "\$COMP" ]]; then
continue
fi
build_component \$COMP \$STOP_ON_ERROR
if [[ \$? -eq 1 ]]; then
exit 1
fi
done < /data/repo/pkg/list_component < \$WORKSPACE/build_components
sort tmpbt.log > build_components
sed -i 's/^[0-9]* //g' build_components
EOT
|
If you’d like stop process on the first error, put 1 the STOP_ON_ERROR variable STOP_ON_ERROR=1
. In other case it will build all components one-by-one regardless of errors. And in every component folder theberror.log
file will be created, where build errors will be stored.
...
Expand |
---|
title | Running with the screen utility |
---|
|
If you are connected by SSH and would like to turn off SSH session or even temporary turn off your computer, then you have enter the following command for tests running: Code Block |
---|
screen ./build_all.sh |
Then you can in any time press Ctrl-A d and after that close your SSH session. When you would like to get back, login again to SSH please and enter the command: |
...
Code Block |
---|
|
cat > check_fails.sh <<EOT
#!/bin/bash
#WDIR=\$1
WDIR=/export/home/du4
subdirs=\$(ls -1 \$WDIR)
for dr in \$subdirs
do
res=\$(tail -1 \$WDIR/\$dr/berror.log)
if [[ "\$(echo \$res | grep 'Fetched')" == "" ]]; then
cat \$WDIR/\$dr/berror.log
fi
done
echo "\$dr"
:echo \$res"
fi
done"================= Local repo sources"
cat /etc/apt/sources.list
EOT
chmod 777 ./check_fails.sh
./check_fails.sh
|
...