Native UUID Generation

I’m checking now Dave. OS X works fine in 64-bit, but standby for Windows and Linux. I’ll post the full code when done.

Thanks Kem…

Thom… I want to use the platform specific methods where possible, and fall back (as Kem did in his example), only if the other methods are not available

Windows 64-bit seems fine…

And Linux too. Here is the complete code. Note the commented “MsgBox” line. That’s how I confirmed on all platforms that the manual code was not running.

Protected Function GenerateUUID() as String
  // Tries to use declares to let the native system functions handle this.
  // Otherwise, falls back to manual creation.
  
  dim result as string
  
  dim useDeclares as boolean = true
  
  try
    
    #if TargetMacOS
      
      soft declare function NSClassFromString lib "Cocoa" ( clsName as cfstringref ) as ptr
      soft declare function UUID lib "Cocoa" selector "UUID" ( clsRef as ptr ) as ptr
      soft declare function UUIDString lib "Cocoa" selector "UUIDString" ( obj_id as ptr ) as cfstringref
      
      dim classPtr as Ptr = NSClassFromString( "NSUUID" ) 
      if classPtr = nil then
        useDeclares = false
      else
        dim NSUUID as ptr = UUID( classPtr )
        
        result = UUIDString( NSUUID )
      end if
      
    #elseif TargetWindows
      
      const kLibName = "rpcrt4"
      
      if not System.IsFunctionAvailable( "UuidCreate", kLibName ) then
        useDeclares = false
      elseif not System.IsFunctionAvailable( "UuidToStringA", kLibName ) then
        useDeclares = false
      else
        soft declare function UUIDCreate lib kLibName alias "UuidCreate" ( ByRef uuid as WindowsUUID ) as Integer
        soft declare function UUIDToString lib kLibName alias "UuidToStringA" ( ByRef inUUID as WindowsUUID, ByRef outString as CString ) As Integer
        
        dim uuid as WindowsUUID
        if UUIDCreate( uuid ) <> 0 then
          useDeclares = false
        else
          dim out as CString
          if UUIDToString( uuid, out ) <> 0 then
            useDeclares = false
          else
            result = out
            result = result.DefineEncoding( Encodings.UTF8 )
            result = result.Uppercase
          end if
          
        end if
      end if
      
    #elseif TargetLinux
      
      const kLibName = "uuid"
      
      if not System.IsFunctionAvailable( "uuid_generate", kLibName ) then
        useDeclares = false
      elseif not System.IsFunctionAvailable( "uuid_unparse_upper", kLibName ) then
        useDeclares = false
      else
        soft declare sub UUIDGenerate lib kLibName alias "uuid_generate" ( ByRef uuid as LinuxUUID )
        soft declare sub UUIDUnparse lib kLibName alias "uuid_unparse_upper" ( ByRef uuid As LinuxUUID, ByRef out As LinuxUUIDString )
        
        dim uuid as LinuxUUID
        UUIDGenerate( uuid )
        
        dim out as LinuxUUIDString
        UUIDUnparse( uuid, out )
        
        result = out.Data
        result = result.DefineEncoding( Encodings.UTF8 )
      end if
      
    #else
      useDeclares = false
    #endif
    
  catch err as RuntimeException
    useDeclares = false
    if err IsA EndException or err IsA ThreadEndException then
      raise err
    end if
  end try
  
  if not useDeclares then
    //
    // Fallback to manual creation
    //
    // From http://www.cryptosys.net/pki/uuid-rfc4122.html
    //
    // Generate 16 random bytes (=128 bits)
    // Adjust certain bits according to RFC 4122 section 4.4 as follows:
    // set the four most significant bits of the 7th byte to 0100'B, so the high nibble is '4'
    // set the two most significant bits of the 9th byte to 10'B, so the high nibble will be one of '8', '9', 'A', or 'B'.
    // Convert the adjusted bytes to 32 hexadecimal digits
    // Add four hyphen '-' characters to obtain blocks of 8, 4, 4, 4 and 12 hex digits
    // Output the resulting 36-character string "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
    //
    
    #pragma BackgroundTasks False
    #pragma BoundsChecking False
    #pragma NilObjectChecking False
    
    System.DebugLog CurrentMethodName + ": Generating manually"
    'MsgBox "Manual!"
    
    dim randomBytes as MemoryBlock = Crypto.GenerateRandomBytes( 16 )
    randomBytes.LittleEndian = false
    dim p as Ptr = randomBytes
    
    //
    // Adjust seventh byte
    //
    dim value as byte = p.Byte( 6 )
    value = value and CType( &b00001111, Byte ) // Turn off the first four bits
    value = value or CType( &b01000000, Byte ) // Turn on the second bit
    p.Byte(6) = value
    
    //
    // Adjust ninth byte
    //
    value = p.Byte( 8 )
    value = value and CType( &b00111111, Byte ) // Turn off the first two bits
    value = value or CType( &b10000000, Byte ) // Turn on the first bit
    p.Byte( 8 ) = value
    
    result = EncodeHex( randomBytes )
    result = result.LeftB( 8 ) + "-" + result.MidB( 9, 4 ) + "-" + result.MidB( 13, 4 ) + "-" + result.MidB( 17, 4 ) + _
    "-" + result.RightB( 12 )
  end if
  
  return result
  
End Function

And I just updated M_String on my web site to the latest version. It includes this code:

http://www.mactechnologies.com/index.php?page=downloads#m_string

Perfect… that is the version I incorporated… I just wasn’t sure if I needed to change the Target to TARGETWINDOWS (which I guess I do :slight_smile: )

I’m not trying to convince you to use one or the other (no skin off my back either way) but I’m curious why you want it this way. Do you believe that the system declares would produce different results? Isn’t that just more code to maintain, test, and potentially fail? v4 UUIDs are dead simple, and Xojo’s GenerateRandomBytes is a CSPRNG, so what the Xojo code produces should be exactly the same as the system.

I guess another way to frame the question: what am I missing by NOT using the declares?

The system calls are significantly faster, for whatever that’s worth.

Edit: In 32-bit, I haven’t compared 64-bit.

[quote=293493:@Kem Tekinay]The system calls are significantly faster, for whatever that’s worth.

Edit: In 32-bit, I haven’t compared 64-bit.[/quote]
Well there’s value in that.

This works for iOS, btw:

dim result as text

soft declare function NSClassFromString lib "Foundation" ( clsName as CFStringRef ) as ptr
soft declare function UUID lib "Foundation" selector "UUID" ( clsRef as ptr ) as ptr
soft declare function UUIDString lib "Foundation" selector "UUIDString" ( obj_id as ptr ) as CFStringRef

dim classPtr as Ptr = NSClassFromString( "NSUUID" ) 
dim NSUUID as ptr = UUID( classPtr )
result = UUIDString( NSUUID )

return result

Finally, I converted my code to use the new framework entirely (Text, Xojo.Core.MemoryBlock, etc.), and timed 100 calls both through the system and manually in both 32- and 64-bit.

32-bit
Declares   0.0004s
Manual     0.0075s

64-bit (agressive)
Declares   0.0003s
Manual     0.0075s

Not much difference between 32- and 64-bit, and the system calls are roughly 20x faster than manual in each.

(All on a Mac, btw. I didn’t check the other platforms.)

(The timing was done through XojoUnit so there is some additional expense included like validation of the UUID after creation and the XojoUnit framework calls.)

Hey uh… Kem? What are these???

dim uuid as LinuxUUID
dim out as LinuxUUIDString

compiler balks at these datatypes

Ah, those structures that are listed in an earlier post.

WindowsUUID
  Data1 As UInt32
  Data2 As UInt16
  Data3 As UInt16
  Data4 As String * 8

LinuxUUID
  Bytes1 As String * 4
  Bytes2 As String * 2
  Bytes3 As String * 2
  Bytes4 As String * 2
  Bytes5 As String * 6

LinuxUUIDString
  Data As String * 36
  TrailingNull As String * 1

Or just copy and paste out of M_String.

Hey everyone,

I tried to integrate Kem’s Multi-OS function to an Module which uses the new Xojo Framework. I’m not fit in handling with the new MemoryBlock/MutableMemoryBlock. I’ll get an Compiler-Error for these part of the function:

[code]If Not useDeclares Then

//
// Fallback to manual creation
//
// From http://www.cryptosys.net/pki/uuid-rfc4122.html
//
// Generate 16 random bytes (=128 bits)
// Adjust certain bits according to RFC 4122 section 4.4 As follows:
// set the four most significant bits of the 7th Byte to 0100’B, so the high nibble is ‘4’
// set the two most significant bits of the 9th Byte to 10’B, so the high nibble will be one of ‘8’, ‘9’, ‘A’, Or ‘B’.
// Convert the adjusted bytes to 32 hexadecimal digits
// Add four hyphen ‘-’ characters to obtain blocks of 8, 4, 4, 4 And 12 hex digits
// Output the resulting 36-character String “XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX”
//

#Pragma BackgroundTasks False
#Pragma BoundsChecking False
#Pragma NilObjectChecking False

System.DebugLog CurrentMethodName + “: Generating manually”
'MsgBox “Manual!”

Dim randomBytes As MemoryBlock = Crypto.GenerateRandomBytes(16) // ← needs to be Xojo.Core.MemoryBlock, but then the EncodeHex function won’t work. How to modify?

randomBytes.LittleEndian = False

Dim p As Ptr = randomBytes

//
// Adjust seventh Byte
//

Dim value As Byte = p.Byte(6)

value = value And CType(&b00001111, Byte) // Turn off the first four bits
value = value Or CType(&b01000000, Byte) // Turn on the second bit
p.Byte(6) = value

//
// Adjust ninth Byte
//

value = p.Byte(8)
value = value And CType(&b00111111, Byte) // Turn off the first two bits
value = value Or CType(&b10000000, Byte) // Turn on the first bit
p.Byte(8) = value

result = EncodeHex(randomBytes) // ← won’t work with the new Xojo-Framework
result = result.LeftB(8) + “-” + result.MidB(9, 4) + “-” + result.MidB(13, 4) + “-” + result.MidB(17, 4) + _
“-” + result.RightB(12)

End If[/code]

Detail:

Martin, I created a new-framework version in my M_Text module, if you want to take a look.

[quote=293581:@Kem Tekinay]This works for iOS, btw:

[code]
dim result as text

soft declare function NSClassFromString lib “Foundation” ( clsName as CFStringRef ) as ptr
soft declare function UUID lib “Foundation” selector “UUID” ( clsRef as ptr ) as ptr
soft declare function UUIDString lib “Foundation” selector “UUIDString” ( obj_id as ptr ) as CFStringRef

dim classPtr as Ptr = NSClassFromString( “NSUUID” )
dim NSUUID as ptr = UUID( classPtr )
result = UUIDString( NSUUID )

return result
[/code][/quote]

Thanks a ton for this, @Kem Tekinay!
I have a question concerning performance. You’ve shown that the declares route is much faster for record creation. I wondering about the hexadecimal vs integer-UUIDs. In the Filemaker system, it’s been shown that a UUID converted to an integer is significantly better in terms of indexing and search performance (as well as file size). If I’m using UUIDs for my primary key (in SQLite) and nearly every table is related to some other table (using the UUIDs for fk-pk matching), would there be a performance advantage to convert to an integer? I’m most concerned about performance in finding related records.

For some background:
I only have about a dozen tables. Each table is not expected to ever have more than 10-15k records (with an average of 2-3k). Each time a record is loaded from the most used table, I need to display ~5-50 related records from multiple descendant levels. I need the display if these related records to be “instantaneous” as the user flips through parent records.

Thanks for any input!

I don’t know, but this should be easy enough to test, no?

My feeling is it won’t make a difference.