Jul 20 2016

Three ways of batch renaming files

Category: Uncategorizedguillem @ 2:49 PM

I’ve found myself having to change the name of a few files in batch by prepending something to them. Let’s see three easy ways to batch rename:

Universal way, works everywhere (we’re talking *nix OS here)


<span class="kwd">for</span><span class="pln"> filename </span><span class="kwd">in</span> <span class="pun">*.</span><span class="pln">jpg</span><span class="pun">;</span> <span class="kwd">do</span><span class="pln"> mv </span><span class="str">"$filename"</span> <span class="str">"prefix_$filename"</span><span class="pun">;</span> <span class="kwd">done</span><span class="pun">;

</span>
Little less universal but works if you have rename

<span class="pln">rename </span><span class="str">'s/(.*)$/new.$1/'</span><span class="pln"> original</span><span class="pun">.</span><span class="pln">filename

</span>
Lovely way in bash and zsh

<span class="pln">mv </span><span class="pun">{,</span><span class="pln">new</span><span class="pun">.}</span><span class="pln">original</span><span class="pun">.</span><span class="pln">filename</span>

 


Mar 01 2016

squashfs filesystem and linux file capabilities

Category: Uncategorizedguillem @ 4:55 PM

For a project at my current workplace, I had to modify the firmware image of a Ubiquiti device (an EdgeRouter to be precise. Pretty nice device considering its price, worth trying!).

The firmware image itself is just a .tar file which after being unarchived (it’s not actually gzipped or anything, just tar’d) will output a bunch of files. The one I was interested in is squashfs.tmp, which, as the name suggests, is the squashed filesystem for the router. I did unsquash it by just running the unsquashfs command, modified the files I was interested in modifying, mksquashfs again, generate the md5 checksum (there is a file to verify integrity of the firmware image, called squashfs.tmp.md5 which essentially holds an md5 of the squashfs.tmp file, quite simple) and tar it again. After that, just upload to the router, wait for it to update and happy days! -> WRONG! So turns out that the router upgraded successfully to the new image, but when trying to enter into the ‘configure’ mode (as you may find in a Cisco or Juniper device) it would complain with a message saying:

Failed to set up config session

What?? I hadn’t seen that message before. My first idea was to do a ‘sudo su’ on the router (it’s essentially running a debian and you’re free to mess as much as you want; thanks ubiquiti!) and try running ‘configure’ again and this time it did not complain. Mhhh… looks like a permission issue then… Next step, quick Google search (saves a ton of time usually) and I found this post at Ubiquiti forums: Building custom image for EdgeMax Really helpful, because as you may be able to see at the end of the post (just before my comment to help on the poor souls having the same issue) somebody noticed that this was in fact an issue related to extended file attributes, not being properly preserved. Knowing that, I decided to check If that was really the issue. Mounting the squashfs.tmp file directly gave this result:

 # mount -o loop squashfs.tmp/tmp/router-fs-test/ # getcap /tmp/router-fs-test/usr/sbin/ubnt-util # /tmp/router-fs-test/usr/sbin/ubnt-util = cap_net_raw,cap_sys_admin+ep

But then, when unsquashing the filesystem and running getcap, guess what happened?

 # unsquashfs squashfs.tmp # getcap squashfs-root/usr/sbin/ubnt-util # 

Nothing! Nada! HA! So the issue happens to be when we unsquash the filesystem. I made sure to have the options enabled to preserve the xattr of files, which btw are enabled by default on Linux (be careful Mac users, they’re not there by default on Mac OS X builds of squashfs-tools), so clearly the issue was somewhere else. Some more google searching led to this: squashfs-tools: unsquashfs not preserving file capabilities Oh, that may be why… so there is a bug on the unsquashfs, at least on 4.2. I went straight to the squashfs site and got the latest package (4.3 (2014/05/12)) and unsquashed the filesystem again after compiling the tools. Still no luck, so I guess the bug is still there on 4.3. Pretty easy, just edit the file squashfs-tools/unsquashfs.c and find the function:

<p class="p1"><span class="s1">int</span><span class="s2"> set_attributes(</span><span class="s1">char</span><span class="s2"> *pathname, </span><span class="s1">int</span><span class="s2"> mode, uid_t uid, gid_t guid, </span><span class="s1">time_t</span><span class="s2"> time, </span><span class="s1">unsigned</span> <span class="s1">int</span><span class="s2"> xattr, </span><span class="s1">unsigned</span> <span class="s1">int</span><span class="s2"> set_mode)</span></p>
<p class="p1">

(Should be somewhere around line 800).

Remove the call to

</p>
<p class="p1"><span class="s1">write_xattr(pathname, xattr);</span></p>
<p class="p1">

from the top of the function and place it just before the return TRUE statement of the function. Compile again the unsquashfs tool and done! Now the file capabilities are correctly

 


Feb 28 2016

New blog start

Category: Uncategorizedguillem @ 7:34 PM

No, i’m not really planning on having a clean start for the blog. I’m just thinking about bringing it to life again. It’s been a long while since I haven’t done any update and I’d like to keep it more updated with different sets of things. From things I’m technically interested or fascinated by to just random thoughts.

There will probably be also some additional changes. First of all, I’m planning on changing the theme to something a bit more ‘modern’. I’m looking forward to using a theme as clean as possible. Ideally I’d like a theme that resembles the appearance of ghost blogging platform.

The next change will be regarding language. I’ve been writing some articles both in Spanish and English, but I find myself more and more in an english only environment so I guess that I’ll switch to writing only in English unless it’s something really directed to Spanish people.

Last but not least, I expect to be updating my blog at least a couple times a week. I don’t plan to write a full 10 page entry every time (It’s never been my style anyway) but at least try to include two new entries every week, even if those are not very long. I’m not the kind of people that enjoy writing per se, so I always try to go as quickly as possible to the point I’m trying to make.

I hope my very-few readers don’t mind the changes and at the same time, I hope that increasing the number of posts will help to improve my online profile and get me new readers.

Yours truly, Guillem


Feb 09 2014

Partition drive larger than 2TB

Category: Linuxguillem @ 5:35 PM

If you need to create a partition a drive larger than 2TB and you’re facing trouble, on gparted, you need to go to ‘Device’ -> ‘Create partition table’ and expand ‘Advanced’. There, you need to select ‘gpt’ as partition type. After that, you will be able to create any partition with any filesystem (as long as it suppots 4TB).

Tags: , , , , , ,


Jan 04 2012

BlackBerry PlayBook USB Host (connect a pendrive) – How to

Category: BlackBerry,Código,Playbook,pseudo-hacksguillem @ 2:02 AM

Well, I finally found time to write a little howto on using your pendrive with the PlayBook. I’m sorry about the delay on publishing this, but i was out of home for a few days and couldn’t find time to write the howto.

Well, let’s go to the howto. Things you will need:

– BlackBerry PlayBook

– Root access to your PlayBook (thanks to dingleberry :) )

– Female USB A to micro USB cable and a soldering iron OR USB OTG cable (for example this one will work: Micro USB OTG Cable)

– A pendrive

Now i will try to make instructions as short and clear as possible. Just let me know if something is unclear. I would also like to let you know that i have only tested this on a 1.8.xxx PlayBook version, so results on OS2 are unknown (i haven’t got access to a PlayBook with OS2 and i’m not wishing to upgrade mine at the moment). Please keep this in mind when trying to use this method.

First step – The USB OTG Cable

Get a USB OTG cable or just find a Female USB A to micro USB cable, open the micro USB side and solder pins 4 and 5 of the micro USB port together (this is what turns the USB port into host mode). The photo shows the first version i used. Then i decided to get a nice USB OTG angle cable as you can see in the first photo of the post. If you are interested in one of those, just let me know.

Second step – Console commands

When you have your USB OTG cable, you can proceed to run the commands to load the necessary kernel drivers. The nice part is that everything is already on the PlayBook, so you don’t really need to upload anything. The commands you will need to run are the following:

slay RIM_usbmgr-Winchester
slay io-usb
slay devb-umass
sleep 2
RIM_usbmgr-Winchester -m0s
io-usb -domap4430-mg ioport=0x4a0ab000,irq=124
sleep 2
waitfor /dev/io-usb/io-usb 4
devb-umass cam pnp blk automount=+hd6t6:/accounts/1000/shared/usb:dos,automount=+hd6:/accounts/1000/shared/usb:dos

The only line you really need to pay attention to, is the last one. It shows where the pendrive will be automatically mounted after you connect it. You can specify different mount points for different partitions just separating them by comas (in my example command, hd6t6 and hd6 shown)

The other commands just kill the USB server running and start it with the required commands to be able to run the USB port as a host port. There is also a command to load the kernel driver (io-usb domap….).

Last step – Connect your pendrive

If all went well, you should now be able to connect your pendrive and access contents on the mount folder you selected. Please keep also in mind that you must first create this directory for the automount to work. I also noticed that sometimes the kernel modules does produce a bus error and crashes. If that happens, just try to run all the commands again.

I really expect to be able to improve this method or provide a ready app to execute all the process, but at the current time, this has no ETA. If you want, you can follow me on @guillemmateos and i will tweet any finding or progress I make about this.

If you have any suggestions or are able to get different OS versions working with this method I’d be really happy if you could post it here and let me know.

Thanks!

Tags: , , ,


Next Page »