Enhancement of AbstractDistiller

[Former user]

My custom distiller looks like this:

public $keysToKeep = ['uid', 'startDate', 'startTime', 'categories', 'originalObject.abstract', 'originalObject.images.0.publicUrl'];

The dotted keys address arrays normally not availible in pluck(). Thats why I flattened the array using a little recursive function and then you can use the dotted key notation. Call function in pluck() before the foreach loop:

$keyStr='';
//\nn\t3::debug(['$data'=>$data],'data vorher');
$this->flattenArray($data, $data, $keyStr);
//\nn\t3::debug(['$keysToKeep'=>$this->keysToKeep,'$data'=>$data],'data nachher');

new function flattenArray():

private function flattenArray( &$data, &$arr, &$keyStr )
{
    //\nn\t3::debug(['$arr'=>$arr,'$keyStr'=>$keyStr]);

    foreach ($arr as $key => $val)
    {
        if (is_array($val))
        {
            $keyStr .= $key.'.';
            $data[ $keyStr ] = $this->flattenArray($data, $val, $keyStr);
            unset($data[$keyStr]);  // remove unneccesary "aaaa." => NULL entries
            //unset($data[substr($keyStr, 0, -1)]);  // remove now obselete arrays - does not yet work
        } else {
            $data[ $keyStr.$key ] = $val;
        }
    }
    // leave recursion: shorten the string by "abcd.", but do not allow "."
    $rpos = strrpos($keyStr, '.', -2);
    $keyStr = substr($keyStr,0,($rpos?:0)).($rpos?'.':'');
    //\nn\t3::debug(['$rpos'=>$rpos,'$keyStr'=>$keyStr]);
}

[David Bascom]

I really love that idea!

A few thoughts:
If I understand your method right, then your resulting JSON will be something like this, right?

{
  "uid": "...",
  "originalObject.abstract": "...",
  "originalObject.images.0.publicUrl": "..."
}

The key in the JSON is the path defined in $keysToKeep – or am I wrong?
Maybe defining a key/value pair would be better and even “flatter”, e.g.:

public $keysToKeep = ['uid'=>'uid', 'abstract'=>'originalObject.abstract', 'image'=>'originalObject.images.0.publicUrl'];

Which then could result in a JSON that looks like this:

{
  "uid": "...",
  "abstract": "...",
  "image": "..."
}

nnhelpers could help us accomplish this with a few lines of code:

// Example array or nested object
$obj = [
    'uid' => 1,
    'image' => ['url' => 'test'],
];

$helper = \nn\t3::Settings();

// Keys (paths) to fetch
$keys = ['uid'=>'uid', 'image'=>'image.url'];
$flatResult = [];

// First convert to an array (might need to pass depth, see docs)
$arr = \nn\t3::Convert( $obj )->toArray();

// magic
foreach ($keys as $key=>$path) {
    $flatResult[$key] = $helper->getFromPath($path, $arr);
}

‌

[Martin Keller]

This is exact the next logical step!
public $keysToKeep = ['uid'=>'uid', 'abstract'=>'originalObject.abstract', 'image'=>'originalObject.images.0.publicUrl'];
Will do some testing on it…

‌

[Martin Keller]

This is my new function AbstractDistiller->pluck() which works like a charm (no need for recursive function anymore).
BREAKING: this is now working only with the key=>value style keysToKeep, otherwise the resulting JSON misses the keys.

    /**
     * Array auf einzelne Felder reduzieren.
     * Allow dotted keysToKeep
     * ```
     * $this->pluck( $assArr, ['uid'=>'uid', 'publicUrl'=>'originalObject.images.0.publicUrl'] );
     * ```
     * @return void
     */
    public function pluck( &$data = [], $keysToKeep = [] ) {
        if (!$keysToKeep) return;
        $helper = \nn\t3::Settings();
        $flatResult = [];
        foreach ($keysToKeep as $key=>$path) {
            $flatResult[$key] = $helper->getFromPath($path, $data);
        }
        $data = $flatResult;
    }

‌

[David Bascom]

Hey, very cool. This is going to be great.
What about this variation: Allow associative AND standard (numeric) arrays. This would prevent a breaking change and be even more flexible:

    /**
     * Array auf einzelne Felder reduzieren.
     * ```
     * // Simple array: Properties of given keys are returned
     * $this->pluck( $arr, ['uid', 'images', 'title'] );
     *
     * // Associative array: Get deep nested property and map it to a new key
     * $this->pluck( $arr, ['uid'=>'uid', 'publicUrl'=>'images.0.publicUrl'] );
     *
     * // Mixture is also possible
     * $this->pluck( $arr, ['uid', 'title', publicUrl'=>'images.0.publicUrl'] );
     * ```
     * @return void
     */
    public function pluck( &$data = [], $keysToKeep = [] ) {
        if (!$keysToKeep) return;
        $helper = \nn\t3::Settings();
        $flatResult = [];
        foreach ($keysToKeep as $key=>$path) {
            if (is_numeric($key)) {
                $flatResult[$path] = $data[$path];
            } else {
                $flatResult[$key] = $helper->getFromPath($path, $data);
            }
        }
        $data = $flatResult;
    }

‌

[Martin Keller]

Functionality of custom distiller seems to have stopped with 1.40; I only get empty value [] for the most basic distiller

public $keysToKeep = [ 'uid','title','abstract'];

Global distilling using TS still works.

Unfortunately, the dotted notation did it not yet make into the code.

‌

‌

[David Bascom]

Hello Martin!

It must have to do with the conversion on an object to an array in this commit:

https://bitbucket.org/99grad-team/nnrestapi/diff/Classes/Distiller/AbstractDistiller.php?at=9-11&diff2=0c87d15f678d264fc26118ef994347868224bf0f

Could you check if removing the is_object condition will make it work again?

What type of class / object / model are you processing here?

[Martin Keller]

Removing the condition and so always casting data to array does not change results.
My favorite model in this case is calendarize-event with its IRRE sys_file_reference. My other favorite model is calendarize-index where the author gave a nice function getOriginalObject() to access related properties from other models.

[David Bascom]

Hello Martin!

Be my guinea-pig :)

Could you try replacing the Nng\Nnrestapi\Distiller\AbstractDistiller with the following code?
It is also in this commit. The dotted notation should be in this commit, too.

Here is the complete code for the AbstractDistiller-file:

<?php

namespace Nng\Nnrestapi\Distiller;

class AbstractDistiller 
{
    /**
     * Definiert, welche Felder/Keys im Array behalten werden sollen.
     * Wenn leer, wird das komplette Array zurückgegeben.
     * Wird von den einzelnen Distillern überschrieben.
     * 
     * @var array
     */
    public $keysToKeep = [];


    /**
     * Wird von ApiController aufgerufen bevor die Daten zurückgegeben werden.
     * Zentrale Methode zum Bearbeiten / Distillen der Daten.
     * 
     * ```
     * $this->processData( $assArr );
     * $this->processData( [$assArr, $assArr, ...] );
     * ```
     * @return void
     */
    public function processData( &$data = [] ) 
    {
        if (is_a($data, \TYPO3\CMS\Extbase\Persistence\Generic\QueryResult::class)) {
            $data = $data->toArray();
        } else if (is_a($data, \stdClass::class)) {
            $data = (array) $data;
        }

        if ($this->isAssoc( $data )) {
            $this->process( $data );
            $this->pluck( $data, $this->keysToKeep );
        } else {
            foreach ($data as &$row) {
                $this->process( $row );
                $this->pluck( $row, $this->keysToKeep );
            }
        }
    }


    /**
     * Prüft, ob es sich um ein assoziatives Array handelt.
     * ```
     * $this->isAssoc( $arr );
     * ```
     * @return boolean
     */
    public function isAssoc( $arr = [] ) 
    {
        if (array() === $arr) return false;
        return array_keys($arr) !== range(0, count($arr) - 1);
    }


    /**
     * Einzelnes Element bearbeiten.
     * Diese Methode wird von den einzelnen Distillern überschrieben.
     * ```
     * $this->process( $assArr );
     * ```
     * @return void
     */
    public function process( &$data = [] ) {}


    /**
     * Reduce array to single fields.
     * ```
     * // Simple array: Properties of given keys are returned
     * $this->pluck( $arr, ['uid', 'images', 'title'] );
     * 
     * // Deep array: Properties of nested array are returned. Key is returned in dot-syntax
     * $this->pluck( $arr, ['uid', 'images', 'title', 'images.0.publicUrl'] );
     *
     * // Associative array: Get deep nested property and map it to a new key
     * $this->pluck( $arr, ['uid'=>'uid', 'publicUrl'=>'images.0.publicUrl'] );
     *
     * // Mixture is also possible
     * $this->pluck( $arr, ['uid', 'title', publicUrl'=>'images.0.publicUrl'] );
     * ```
     * @return void
     */
    public function pluck( &$data = [], $keysToKeep = [] ) 
    {
        if (!$keysToKeep) return;

        $helper = \nn\t3::Settings();
        $objHelper = \nn\t3::Obj();
        $flatResult = [];

        $arr = $objHelper->toArray($data, 6);
        if (is_object($data)) {
            \Nng\Nnrestapi\Distiller\ModelDistiller::process( $data, $arr );
        }

        foreach ($keysToKeep as $key=>$path) {
            if (is_numeric($key)) {
                $key = $path;
            }
            $flatResult[$key] = $helper->getFromPath($path, $arr);
        }
        $data = $flatResult;
    }

}

‌

[Martin Keller]

Using a model as source, isAssoc() is fed with an object. So array_keys() fails expecting an array as parameter.

public function isAssoc( $arr = [] ) 
    {
        if (array() === $arr) return false;
        return array_keys($arr) !== range(0, count($arr) - 1);
    }

‌

‌

[David Bascom]

Could you give me the code of your endpoint so I can see what exactly is being passed to this method?

[David Bascom]

Fixed in coming release