Missing matrix_float3x3 type for warpTransform selector in Vision.VNImageHomographicAlignmentObservation

Issue #214 duplicate
Emmanuel Piuze
created an issue

The selector warpTransform of the class VNImageHomographicAlignmentObservation is broken in the Vision framework due to what appears to be a missing type (matrix_float3x3):

    from Vision import VNHomographicImageRegistrationRequest, VNSequenceRequestHandler
    request = VNHomographicImageRegistrationRequest.alloc().initWithTargetedCIImage_options_(cimg1, None)
    handler = VNImageRequestHandler.alloc().initWithCIImage_options_(cimg2, None)
    handler.performRequests_error_([request], None)
    print request.results()
    for observation in request.results():
        # unknown type to pyobjc: matrix_float3x3
        # 3 vectors of 3 x 32 bit floating point numbers
        t = observation.warpTransform
        print t
        t()

Output:

(
    "<VNImageHomographicAlignmentObservation: 0x7fe629cf09f0> EE131EE1-BCF3-4443-B1C4-EE64717B8843 1"
)

<native-selector warpTransform of <VNImageHomographicAlignmentObservation: 0x7f91302ac650> 6F4386B6-C77B-4CDC-B5E4-22A2A5468876 1>

objc.internal_error: PyObjCRT_SkipTypeSpec: Unhandled type '0x5d' ]}16@0:8

Comments (8)

  1. Ronald Oussoren repo owner

    There are two problems:

    1) There is no code for matrix_float3x3 in the method signature.

    That's technically an Apple issue: the compiler encodes SIMD types like this as an empty string :-(.

    2) PyObjC currently doesn't support SIMD types at all.

    This is primarily because libffi (the library used to create C stack frames for calling Objective-C methods) doesn't support these types.

    My current plan is to fix both issue with overridden type encoding for SIMD types, with an PyObjC-specific type encoding similar to some other custom type encodings we use, combined with custom wrappers for calling these methods.

    That should reduce the amount of work needed to get full support for APIs with SIMD types, but is still significant work.

  2. Emmanuel Piuze reporter

    I see, thanks for the update. Is there any “reasonable” workaround that comes to mind? For the time being I will be forced to run the vision code in ObjC upstream, serialize to disk, and then reload from my python script.

  3. Ronald Oussoren repo owner

    The only somewhat simple workaround is to call these ObjC APIs from Objective-C. One way to do this is to write categories on the classes you use that add selectors that have arrays of basic types as arguments instead of SIMD types and then call the real method.

  4. Log in to comment