How to access data on an unmounted partition

Hello,

Recently I was able to install a new, 4x larger, hard drive in my DVD/hd-recorder. The old one still worked flawlessly after 8 years (happy me !), but was almost full.

As the menus on the recorder look Linux-like and hardware now is often Linux-based, I tried to mount this old SATA hard drive on my Linux-notebook,
in order to archive the files on it. GParted however showed it to be of type A5 (FreeBSD).

It should have been possible to mount this partition /dev/sdb1 (spanning the full disk /dev/sdb) by the Linux UFS filedriver. However, this failed. I even
tried a live FreeBSD CD for copiing, but also no luck there. Some further investigation showed that these drives still have Big Endian byte-order, while
nowadays most use (INTEL) Little Endian byte-order. Surprisingly, even some recent models (2013) of other brands recorders apparently still use a similar
engine and old BSD-format. My old PowerPC iMac probably could have accessed the drive…

On the Web there appear to have existed utilities for this Endian-conversion on BSD but, unfortunately, the links are dead or empty now.
Other people were not successfull here, too.

So it will be necessary on Linux to read the partion and reverse the byte-order, which is simple, and then write the blocks to a new BSD-partition elsewhere.  After doing that the filesystem should be visible and accessible there.

  Does anyone know how to access an (unmounted) partition from Linux, read-only. Does one have to use low-level ioctl -functions e.g. for reading ?

You certainly did make a backup before removing the old disk, didn’t you? Copy the data from the backup to the new disk.

Thanks, Eli,

Unfortunately, these models do not have the option to backup to a hard disk (there is only one Sata-connector internally, besides the DVD-drive one) . I did make copies of most programs to DVD, but some were to large to fit on even DL-DVD’s, so the quality had to be adapted (lowered) before copiing.

don’t you have to format the new drive before any attempt to mount it ? (and then only copy datas from the old to the new)

Thanks, Jean-Yves,

Yes, the new drive had to be formatted. One has to use a special “service-remote-control” or the codes used in it available on the Logitech site e.g. That worked perfectly. So it is working OK.

I would like to archive the (ca. 700 !) files on the hard drive to a PC-filesystem, however, since the new drive, as being formatted BSD also, cannot be used (both cannot be mounted, as it is).

sudo mount -r -t ufs -o ufstype=ufs2 /dev/sdb1/ /mnt

I suspect that the box was running a MIPS chipset. Unfortunately, I don’t know of any filesystem tools that will let you mount a big-endian volume onto a little-endian system.

I did a quick chat with one of my FreeBSD contacts and they’re more surprised that the drive’s still operational than anything else. They pointed me to a project that allows you to create a MIPS emulation environment using QEMU and by custom-building a MIPS version of FreeBSD. I’ll post the link, but that’s as much data as I have:

FreeBSD MIPS Emulation using QEMU

My free disk editor iBored, which should also work on Linux, can read and write disks. You could use it to copy the data to a file (menu Disk -> Write Blocks to File…) and then edit the data in the file with your tools of choice, and then write that back to the disk in the end (Write File to Blocks…).

If you want to r/w directly yourself, simply open the sdX drive and read/write using the low level POSIX calls (open, write, read etc.) using declares. But you need to run your app with root permissions, probably. While developing the app, you could launch the entire Xojo IDE as root, making this easier.

Thanks Eli.
Your command sequence is little bit different from the one I tried and more extensive.

Unfortunately the partition is still not recognized (“wrong fs type …”). I think, because of the “wrong” byte order, no file system can be detected. GParted has an option to seek for filesystems, but after a very long run (almost a day …) nothing was detected. The drive still works when re-inserted into the recorder, though.

Thanks, mr. Jones,

An interesting link!

I will have a more thorough look later.

In principle “bare” reading of a drive is possible. The OS can do it and also tools like ddrescue.

Someone else had a project earlier to read badly damaged drives from these type of recorders;
even the partition type (BSD) was unreadable. First the drive was copied (by safe-reading) to a data-partition
and then undamaged mp3-blocks where assembled (sometimes manually) to files, after byte-reversing the data.
So some files could be rescued. Note however that the file-structure was unclear, because the datapartition
was not of type BSD.

Regarding quality:
The system seems to have a clever auto-repair system. Once in a while the recorder starts up with a “repair” message;
this takes a few seconds, but after that the system is fully usable. This has been the case from the start of usage !

Hello Thomas,

Sounds very promising!!

I will give this a try as soon as possible.

Thanks!

Error trying to mount FreeBSD/UFS partition from FreeNAS

You might wanna check software from R-Studio tools for that too, works on any major OS nowadays. Works fine for restoring data from RAID-volumes too. Trial versions available.

You can properly also use: testdisk (free open source: http://www.cgsecurity.org/wiki/TestDisk ) can be found on a single boot cd “system-rescue-cd”: https://www.system-rescue-cd.org

Very good, has recovered my files from a raid5 system.

You are missing the big issue here - the filesystem on the disk is a big-endian filesystem created on an embedded version of FreeBSD running on a MIPS processor. He must use a big-endian OS to read the data back successfully without the need to write byte swapping routines and dig through the disk sector by sector.

While the tools mentioned above work fine in the little-endian world, they won’t do anything with this particular disk.

Building the QEMU environment for FreeBSD MIPS and attaching the drive to that virtualized OS that I linked to above is truly the only straightforward solution.

Thanks, mr. Rottensteiner,

I will have a look at that option too.

I am now investigating the possibilites …

Thanks, mr. Hansen,

Earlier on I had a look at testdisk. However this doesn’t solve the problem of
the reversed byte-order, directly.

Also, the drive seems to be in very healthy condition (there is on option on
the service remote to do extensive testing), so no need to save the partition.

Thanks Tim,

You might be right, because the other options still need a conversion-step (and
one has to start at the right position with inverting, although this info is easily
available in info regarding drive-structures …).

Another remark regarding quality : also it's predecessor Sony model, in a very

nice silvercolor casting, worked flawlessly for more then six years; it didn’t move
with me to my new location, alas. Already good in DVD-production, the newer model
has almost no errors (guess: less than 1 in 50).

The new model is based, internally, on a Pioneer engine. Those Pioneer-models
themselves are, however, much more difficult (if possible at all …) to upgrade;
one would need, besides the (same) service remote, a dedicated service CD, which seems
very difficult to acquire and one would have to change the system-identity precisely
to one allowed in a table, otherwise making the unit useless indefinitely…

The robustness of the recorder is even more noticeable, as, in the first seven years,
much disk-activity was “wasted” on the extremely slow GUIDE+ -system acquiring
data from a TV-channel with program-info, which could not be turned of. A few
years ago there was a Sony firmware-upgrade mostly related to Guide+;
unfortunately, this didn’t add an option to turn it off either. I even asked Sony
about it, but they had no solution.
Later I found out by trying that switching the GUIDE+ -setting from “analog”
to “digital” made it silent …

My rather long answer to my own question has somehow disappeared.

Sorry for the confusion. I will retry later.

There seems to be a problem with long text. So I’ll divide my answer in 3 short sections and post separately.

Part 1:

Unfortunately there were indications that the BSD-signature on the disks to examine would not necessarily lead to a
BSD-type filesystem, like UFS, so the excellent suggestion by @Tim Jones to use a Big-Endian variant of BSD on a MIPS-emulator
might be less usable as hoped originally.

Also, after examining the disk with @Thomas Tempelmann 's IBored program, the newer version of the recorder didn’t expose
the filenames of the recorded programs (possibly encrypted now) unlike the earlier models. But some “magic numbers” pointing to VOB-files were visible.

 However, even in Windows it appears not to be too difficult to access, especially read, a disk that is not yet "mounted",

in “Linux-speak”. I use a 3-step process as described late in this post.

Whether or not this will succeed depends also on the condition/health of the drive and it’s bootsector.
If the drive shows up in Windows’ DiskManagement tool, say as Disk1, one has good chance to be able to acquire a device handle
(even for an already visible, say Windows) drive. If the drive has some serious defects this may not be possible, however.

Reading in general is possible, even the “raw” info of a disk. But to do this, the program you create will in general have to run
with “administrator” permissions; this can be easily achieved by right-clicking your exe-file and choose “run ad administrator”. Of course, in the test-phase one can start Xojo itself with Administrator-priviliges.

NOTE: Reading from a severely damaged drive should be avoided at all cost, as it could deteriorate the drive beyond repair;
leave it to the professionals …

If you only have to look at a disk on a incidental basis, it would be easier to use a disk-viewing application,
Thomas Tempelmann’s IBored allowing you to view the blocks of the drive, copy blocks or sections to your own pc etc. and
search for specific patterns on the drive; in some situations editing is possible too. Also, IBored is scriptable.

However, sometimes it might be handier to access a non mounted drive directly from your program. I will give a small example
here for the necessary systemcalls; of course similar functions for Mac OS and Linux exist, but I have not investigated.
For Linux, it seemed more difficult to retrieve the drive handle for a disk.

First of all, for reading (and also: writing) disk on Windows, one has to know the sectorsize, in bytes.

In general this appears to be (an integer multiple of) 512, but one can inquire the drive by using one of the device
related DeviceIoControl functions, namely the one that takes the parameter IOCTL_DISK_GET_DRIVE_GEOMETRY.
All the control codes can be found at MSDN or, sometimes, in the file winnt.h (in many cases already on your hard disk).
Unlike reading from or writing to a (unmounted) disk, you do not need Administrator permission for this inquiry.

1. Getting a device-handle for the disk

First one has to acquire a so called device handle (integer value) for the drive to be used.
One uses the function CreateFileA, from kernel32.dll; when finished, later the device should be closed.

[code]Declare Function CreateFileA Lib “kernel32” (lpFileName AS CString, dwDesiredAccess AS INTEGER, dwShareMode AS INTEGER, lpSecurityAttributes AS PTR, dwCreateDisposition AS INTEGER, dwFlagsAndAttributes AS INTEGER, hTemplateFile AS INTEGER) AS INTEGER

Declare Function CloseHandle Lib “kernel32” (handle AS INTEGER) AS BOOLEAN
[/code]

Inquiring info from a drive only needs a minimal permission.

Define as follows: DIM Inq_Access AS INTEGER and give it the value 0.
But first look if Xojo can access/find the necessary dll:

IF System.IsFunctionAvailable( "DeviceIoControl", "kernel32.dll" ) THEN
  MsgBox("Function DeviceIoControl is available")
ELSE
  MsgBox("Function DeviceIoControl is NOT available")
END IF

If available, then get the handle for the device. In the case that the drive you are interested in showed up as “Disk1” in
DiskManagement, it will be known as PhysicalDrive1 to the system, to be encoded in a special format: “\\.\PhysicalDrive1”.
In a system with only one hd that built-in drive is known as PhysicalDrive0.

DIM hDisk AS INTEGER

CONST Inq_Access       = 0
CONST INVALID_HANDLE   = -1
CONST File_Share_Read  = &h00000001
CONST File_Share_Write = &h00000002
CONST Open_Existing    = &h00000003
CONST NullHandleTemplateFile = 0       

hDisk = CreateFileA("\\\\.\\PhysicalDrive1", Inq_Access, File_Share_Read, NIL,  Open_Existing, 0, NullHandleTemplateFile)

IF (hDisk = INVALID_HANDLE) THEN
    MsgBox("Device could not be opened")
ELSE
    MsgBox("Device succesfully opened; device-handle = " + STR(hDisk))
END IF

Note: CreateFileA can also be used to open “normal” files, with the first parameter then represented by the
full, classic, path (like “C:\users\ABCD\test.txt” e.g.).

If succesfull, the next step is to determine the number of bytes per sector of the device.