Stuff that has not yet gone into the official build.
Post a reply

Re: Alternative usb installation method, part 2

Fri Nov 22, 2013 12:53 pm

This will do a better job of creating the menu entry. The one in the deb does it wrong.

Below is revised /usr/bin/mkloopfile.sh
Size is set to 100MB for testing, encryption off. You get to select the folder that will contain the loopfile and the custom initrd to be used with it. More choices will come in later rewrites. Haven't figured out the best way to do that yet.

I think this works now. (also fixed the error log location in this one)
Code:
#!/bin/bash
set -x
# scriptname: mkloopfile

# make a loopback filesystem (luks or not)

# use for live-persistence or whatever else.

########################################################
###### EDIT TO SUIT ####################################

# size of the loopback file in MB
#SIZE="500"
SIZE="100"
# if it will be used for persistence
PERSISTENCE="yes"

# name of the loopback file.. (if for persistence) it *should* be labelled "persistence"
# except where "persistence-label=LABEL" is used
# else choose another name
#LOOP_FILENAME="persistence"

# make the loopback file user-owned (uncomment and edit next line)
NORMAL_USER_ID="1000"

# filesystem format
# will be labelled the same as $LOOP_FILENAME
FILESYSTEM="ext2"

# choose "full" or "home".. Ignored if $LOOP_FILENAME is not "persistence"
# other persistence options: choose either and edit.conf manually
PERSISTENCE_TYPE="home"

# mountpoint for the loopback file (script can create it)
LOOP_MOUNTPOINT="/tmp/loopmount"

# Use LUKS container (yes/other)
IS_LUKS="no"


########################################################
########################################################

source /usr/lib/refracta2usb/functions_r2u
yad_zenity_compat

if [ "$1" = "-debug" ]; then
set -x
fi

error_log="/var/log/errors_mkloopback"
exec 2>"$error_log"

# script must be called from a root terminal
if ! [ -t 0 ]; then
echo "Error:  Not a terminal"
exit 1
fi

if ! [[ $(id -u) -eq 0 ]]; then
echo "Error: You are not root"
exit 1
fi


device=$(cat /tmp/r2u_device)
usb_mountpoint="/media/${device##*/}1"

work_dir=$($DIALOG --file-selection --directory --filename="${usb_mountpoint}/*" \
   --text=" Select the folder to contain the loop file. Normally, this is the root
directory of the device. For a multiboot device, choose the folder
for the system that will use this loop file.
" \
   --width=500 --height=400 --${BUTTON0}="OK"${BUTTON0NUM} --${BUTTON1}="Quit Task"${BUTTON1NUM})
   if [[ $? = 1 ]] ; then
      exit 0
   fi
cd "$work_dir"

finished_message="work_dir is $work_dir.
current directory is $(pwd)"
finished_dialog

LOOP_FILENAME=$($DIALOG --entry --entry-text="persistence" --text="Enter a name for the loop file,
or just ENTER to use \"persistence\"."  --${BUTTON0}="OK"${BUTTON0NUM} --${BUTTON1}="Quit Task"${BUTTON1NUM})
   if [[ $? = 1 ]] ; then
      exit 0
   fi

setup_file () {
dd if=/dev/zero of=${LOOP_FILENAME} bs=1M count=${SIZE}

   if [ -n "$NORMAL_USER_ID" ]; then
   chown ${NORMAL_USER_ID}:${NORMAL_USER_ID} ${LOOP_FILENAME}
   fi

losetup -f > /tmp/nextloop
LOOPDEV=$(cat /tmp/nextloop)

losetup ${LOOPDEV} ${LOOP_FILENAME}

}

format_file () {

if [ "$IS_LUKS" = "yes" ]; then
cryptsetup luksFormat ${LOOPDEV}
cryptsetup luksOpen ${LOOPDEV} ${LOOP_FILENAME}

LOOP_FS="/dev/mapper/$LOOP_FILENAME"
CLOSE_LOOPDEV="cryptsetup luksClose $LOOP_FILENAME && losetup -d $LOOPDEV"

else
LOOP_FS="$LOOPDEV"
CLOSE_LOOPDEV="losetup -d $LOOPDEV"

fi

mke2fs -t ext2 -L ${LOOP_FILENAME} ${LOOP_FS}
/sbin/tune2fs -c 0 ${LOOP_FS}

}

mount_fs () {

mkdir ${LOOP_MOUNTPOINT} || echo "Mountpoint exists"

   DIR_FILES=$(ls ${LOOP_MOUNTPOINT}|wc -l)
   if [ "$DIR_FILES" -ne "0" ]; then
   echo "Error: Mountpoint ${LOOP_MOUNTPOINT} is not empty" && exit 1
   fi

mount ${LOOP_FS} ${LOOP_MOUNTPOINT}

}

write_persistence_conf () {

if [ "$PERSISTENCE" = "yes" ]; then

echo -e "# home persistence example:"  > ${LOOP_MOUNTPOINT}/persistence.conf
echo -e "# /home bind,source=.\n" >> ${LOOP_MOUNTPOINT}/persistence.conf

echo -e "# full persistence example (entire filesystem writable):"  >> ${LOOP_MOUNTPOINT}/persistence.conf
echo -e "# / union,source=.\n"  >> ${LOOP_MOUNTPOINT}/persistence.conf

   if [ "$PERSISTENCE_TYPE" = "home" ]; then
   echo -e "/home union,source=." >> ${LOOP_MOUNTPOINT}/persistence.conf
   fi

   if [ "$PERSISTENCE_TYPE" = "full" ]; then
   echo -e "/ union,source=." >> ${LOOP_MOUNTPOINT}/persistence.conf
   fi
fi

}

cleanup () {

#echo "Umount and close now? (y/n)" && read RESP

#if [ "$RESP" = "y" ]; then
while true ; do
   echo " Unmount and close now? (y/n)
"
   read ans
   case $ans in
      y)   umount ${LOOP_FS}
         ${CLOSE_LOOPDEV}
         rm -f /tmp/nextloop
         rmdir /tmp/loopmount
         break ;;
#else
      n)
echo -e "\n ${LOOP_FS} is still mounted at ${LOOP_MOUNTPOINT}\n"
echo -e " To umount and close manually:"
echo -e " umount ${LOOP_FS} "
echo -e " ${CLOSE_LOOPDEV}"
         break ;;
#fi
   esac
done
LOCATION="$(pwd)"
echo -e "\n Loopback File was written to ${LOCATION}/${LOOP_FILENAME}"

}





setup_file
format_file
mount_fs
write_persistence_conf


source_initrd=$($DIALOG --file-selection --height=550 --width=650 --title="$TITLE" \
--text=" Select the custom initrd that will be used with this loopfile.\n" \
--filename="$source_dir")
custom_initrd="${source_initrd##*/}"

if [[ $work_dir = "$usb_mountpoint" ]] ; then
   warning_message="work_dir = usb_mountpoint
   $work_dir = usb_mountpoint "
   live_dir="/live"
else
   live_dir="/${work_dir##*/}/live"
fi

finished_message="
Persistent loopback file is ready.
If you select Edit at the Main Menu, an entry
for this file will be added to your boot menu. "
   echo "$finished_message"

   if [[ $LOOP_FILENAME = "persistence" ]] ; then
      menu_text="
label persistence
   menu label Refracta (persistence)
    kernel ${live_dir}/vmlinuz quiet
    append initrd=${live_dir}/$custom_initrd boot=live ip=frommedia union=aufs persistence
"
   else
      menu_text="
label $LOOP_FILENAME
   menu label Refracta ($LOOP_FILENAME)
    kernel ${live_dir}/vmlinuz quiet
    append initrd=${live_dir}/$custom_initrd boot=live ip=frommedia union=aufs persistence config=openssh-server persistence-label=$LOOP_FILENAME
"
   fi
   
   echo "$menu_text" > /tmp/boot_menu_text
#   chmod 666 /tmp/boot_menu_text
   username=$(cat /tmp/r2u_user)
   chown "$username" /tmp/boot_menu_text
   
finished_dialog

umount ${LOOP_FS}
${CLOSE_LOOPDEV}
rm -f /tmp/nextloop
rmdir /tmp/loopmount

exit 0

Re: Alternative usb installation method, part 2

Fri Nov 22, 2013 7:00 pm

No, the menu entry for the loopback file is still wrong, but you get the idea. Stuff is missing, so I can't get it to mount the persistent loop file. I do seem to have multi-boot working ok, without persistence. (Haven't tried it with a persistent partition yet.) I'll have to play with that some more and take notes. A couple of times I booted to a login screen, but I couldn't type anything to log in.

Re: Alternative usb installation method, part 2

Sat Nov 23, 2013 12:37 pm

The menu entry (I know it's not working yet) says "Loopback 07 Create persistence loopback file. (Requires patched initrd)"

Just to clarify:

Use of persistence files should be supported in wheezy/sid without any modifications to initrd or live-boot

As long as it's not on the live-media partition and not ntfs. The partition label does not matter. Live-boot docs say LUKS can be used but it doesn't actually work.

The purpose of the custom initrd is (only) to allow LUKS files and RW mount of the live-media partition (meaning the same partition can be used for persistence files)

Re: Alternative usb installation method, part 2

Sat Nov 23, 2013 3:33 pm

Looking at the "multiboot" option: syslinux.cfg (or whatever other config file it includes or chains) must point to where are the live files. The "/live" directory then becomes irrelevant, it is only where live-boot will look for the squashfs if no path is specified.

Example: my_snapshot live files are in /my_snapshot (initrd.img, vmlinuz, filesystem.squashfs)

Code:
label my_snapshot
menu label my_snapshot
kernel /my_snapshot/vmlinuz quiet
append initrd=/my_snapshot/initrd.img boot=live other_opts union=aufs live-media-path=/my_snapshot/

There is more than one way to do that, e.g. include a submenu (or chain another syslinux.cfg). You still need some entry in the main menu.

Re: Alternative usb installation method, part 2

Sun Nov 24, 2013 2:52 am

I've got it working sometimes. Right now, it only does persistent looopback files on the live media partition, using custom initrd. Seems to work every time with filename of persistence, but only sometimes with a different filename. These all work:
Code:
label persistence
   menu label Refracta (persistence)
    kernel /live/vmlinuz quiet
    append initrd=/live/initrd.custom.img boot=live ip=frommedia union=aufs live-media-path=/live persistence  basemountmode=rw,noatime,umask=000

label persistence
   menu label Refracta (persistence)
    kernel /sidfracta/live/vmlinuz quiet
    append initrd=/sidfracta/live/initrd.custom.img boot=live ip=frommedia union=aufs live-media-path=/sidfracta/live persistence persistence-path=/sidfracta/  basemountmode=rw,noatime,umask=000

label loop6
   menu label Refracta (loop6)
    kernel /sidfracta/live/vmlinuz quiet
    append initrd=/sidfracta/live/initrd.custom.img boot=live ip=frommedia union=aufs persistence config=openssh-server live-media-path=/sidfracta/live basemountmode=rw,noatime,umask=000 persistence-path=/sidfracta/ persistence-encryption=none,luks persistence-label=loop6


This one doesn't work. The entry is identical to the one for loop6, except the name was replaced with myloop.
Code:
label myloop
   menu label Refracta (myloop)
    kernel /sidfracta/live/vmlinuz quiet
    append initrd=/sidfracta/live/initrd.custom.img boot=live ip=frommedia union=aufs persistence config=openssh-server live-media-path=/sidfracta/live basemountmode=rw,noatime,umask=000 persistence-path=/sidfracta/ persistence-encryption=none,luks persistence-label=myloop


The one for loop6 also works if "persistence-encryption=none,luks" is removed, and that has the added benefit of not asking me for the password for all the encrypted partitions on my hard drives.

Here's the directory listing for the root of the usb drive. testfile6 is just a text file, and persistence is a loopback file.:
Code:
deb_netinstall  pkglist_refracta_7.2_i386-20131020_0352  syslinux
live            Release_Notes                            testfile6
persistence     sidfracta

And this is what's in the sidfracta directory.
Code:
live  loop6  myloop  persistence  pkglist_sid-refracted-20131107_1123  syslinux

Re: Alternative usb installation method, part 2

Sun Nov 24, 2013 2:27 pm

2GB pen. One FAT partition (label r2u) and one ext2 (label "verbatim2") Live system on the FAT, original initrd. Persistence file "sidrw" (not luks) in directory "/test" on the ext2. Working menu snip:

Code:
label Sid-snap persistence
menu label Sid-snap persistence
kernel /live/vmlinuz quiet
append initrd=/live/initrd.img boot=live ip=frommedia union=aufs noeject persistence persistence-media=removable-usb persistence-path=/test/ persistence-label=sidrw

* persistence-path=PATH doesn't work without a trailing slash

* "persistence-path=PATH" with "persistence-label=FILENAME" works here, whether the image's filesystem label (don't confuse this with the actual file's name) is "FILENAME" or "persistence"

* Using "noeject" because the prompt to pull the device on shutdown is irritating.

* "persistence-media=removable-usb" means scan removable-usb only (as does "bootfrom=removable-usb") Fixed disks should then not be checked at all, never mind prompt for LUKS keys. Will probably boot faster.

fsmithred, it's unclear whether you are testing LUKS files or not. If not "persistence-encryption=whatever" should not be there.

I'll take a look at LUKS files on different partitions and/or with different filenames/labels later (persistence-label is new here, never used it)

EDIT
Now using custom inird. Replaced "sidrw" with a LUKS file. Image filesystem ext2, labelled "persistence". It works fine.

Code:
label Sid-snap luks-persistence
menu label Sid-snap luks-persistences
kernel /live/vmlinuz quiet
append initrd=/live/initrd.custom.img boot=live ip=frommedia union=aufs noeject persistence persistence-media=removable-usb persistence-path=/test/ persistence-label=sidrw persistence-encryption=none,luks


After changing the filesystem label to "sidrw" it still works.
Last edited by dzz on Sun Nov 24, 2013 3:13 pm, edited 1 time in total.

Re: Alternative usb installation method, part 2

Sun Nov 24, 2013 3:12 pm

Was testing both luks and non-luks. I think I was writing about non-luks above. Tried it with and without "persistence-encryption=none,luks".

Today's tests (with stock initrd, no luks files):
Copied persistence loopback file (500MB) to root of second partition. Partition not labeled "persistence" but filename is. (So is the filesystem inside the loop file. Using your functions unchanged.)
Boot with "persistence" in cmdline -> whole second partition is mounted on /lib/live/media/persistence/sde2/
With "persistence-storage=file" get same result.
With "persistence-path=/persistence" get no persistence, just regular ro boot.

OK, I noticed in other tests that the trailing slash needs to be on persistence-path, so instead of pointing to the file, I should just point to the directory that contains the file. In the above case, it's in the root directory of the second partition, so I don't need to use "persistence-path=/", right?

Re: Alternative usb installation method, part 2

Sun Nov 24, 2013 3:19 pm

See my "edit" previous post.. I got it working consistently, luks (using custom initrd) or not, in second partition.

If it's in a partition root "persistence-path" should not be used. I don't know "persistence-storage" option, never used it.

If there's a mount in /lib/live/media/persistence/ you know it's working straight away. You can open it and watch as changes get written.

Re: Alternative usb installation method, part 2

Sun Nov 24, 2013 4:39 pm

Found the problem - after I moved some code around, PERSISTENCE="yes" became PERSISTENCE="" which resulted in no persistence.conf being copied to the loopmount. Just fixed it manually and will test now, with stock initrd and with custom. Still no encryption.

Edit: To clarify - /dev/sde2 was being mounted at /lib/live/mount/persistence/sde2, but it didn't contain the user's home. And any files I saved in my home were lost on reboot. Files could be saved to the mount. After fixing the problem, /dev/sde2 is still mounted in the same place, but there's also loop1 mounted at /home.
Last edited by fsmithred on Sun Nov 24, 2013 6:23 pm, edited 1 time in total.

Re: Alternative usb installation method, part 2

Sun Nov 24, 2013 5:49 pm

It works. Thanks. Did it with non-luks persistent loopback on second partition using stock and custom initrd, and did it with luks persistent loopback (and custom initrd) on first partition. Right now, the script will only put the file on the first partition. I'll add support for second (or other) partition. I'm pretty sure most of the code for that is already written - I just have to pull it from the other scripts.

Also need to work on generating menu entries appropriate for the different choices. Current version (I'll upload 9.0.4c soon) will put the loop file in the root of the first partition, or if you have a multiboot, it'll put it in the same directory as the system that will be using it. You get to choose where to put it, but the way the menu entry gets generated, it should be the same as the folder holding the system. If that's not flexible enough, let me know. Or anything else.


Edit: wget this!
http://distro.ibiblio.org/refracta/file ... 0.9.4d.deb

Edit2: updated the above link
Last edited by fsmithred on Mon Nov 25, 2013 3:45 am, edited 1 time in total.
Post a reply