Passing pointer to pointer to float from Java through JNA to a C dynamic library

Dmitri Lihhatsov

Catboost offers a dynamic C library, which theoretically can be used from any programming language.

I'm trying to call it through Java using JNA.

I'm having a problem with the CalcModelPrediction function, defined in the header file as follows:

EXPORT bool CalcModelPrediction(
    ModelCalcerHandle* calcer,
    size_t docCount,
    const float** floatFeatures, size_t floatFeaturesSize,
    const char*** catFeatures, size_t catFeaturesSize,
    double* result, size_t resultSize);

In Java, I've defined the interface function as follows:

public interface CatboostModel extends Library {
        public Pointer ModelCalcerCreate();
        public String GetErrorString();
        public boolean LoadFullModelFromFile(Pointer calcer, String filename);
        public boolean CalcModelPrediction(Pointer calcer, int docCount,
                PointerByReference floatFeatures, int floatFeaturesSize,
                PointerByReference catFeatures, int catFeaturesSize,
                Pointer result, int resultSize);
        public int GetFloatFeaturesCount(Pointer calcer);
        public int GetCatFeaturesCount(Pointer calcer);
    }

and then I'm calling it like this:

CatboostModel catboost;
Pointer modelHandle;

catboost = Native.loadLibrary("catboostmodel", CatboostModel.class);
            modelHandle = catboost.ModelCalcerCreate();
if (!catboost.LoadFullModelFromFile(modelHandle, "catboost_test.model"))
{
    throw new RuntimeException("Cannot load Catboost model.");
}

final PointerByReference ppFloatFeatures = new PointerByReference();
final PointerByReference ppCatFeatures = new PointerByReference();
final Pointer pResult = new Memory(Native.getNativeSize(Double.TYPE));

float[] floatFeatures = {0.5f, 0.8f, 0.3f, 0.3f, 0.1f, 0.5f, 0.4f, 0.8f, 0.3f, 0.3f} ;
String[] catFeatures = {"1", "2", "3", "4"};
int catFeaturesLength = 0;
for (String s : catFeatures)
{
    catFeaturesLength += s.length() + 1;
}

try
{
    final Pointer pFloatFeatures = new Memory(floatFeatures.length * Native.getNativeSize(Float.TYPE));
    for (int dloop=0; dloop<floatFeatures.length; dloop++) {
        pFloatFeatures.setFloat(dloop * Native.getNativeSize(Float.TYPE), floatFeatures[dloop]);
    }
    ppFloatFeatures.setValue(pFloatFeatures);

    final Pointer pCatFeatures = new Memory(catFeaturesLength * Native.getNativeSize(Character.TYPE));
    long offset = 0;
    for (final String s : catFeatures) {
        pCatFeatures.setString(offset, s);
        pCatFeatures.setMemory(offset + s.length(), 1, (byte)(0));
        offset += s.length() + 1;
    }
    ppCatFeatures.setValue(pCatFeatures);

}
catch (Exception e)
{
    throw new RuntimeException("Couldn't initialize parameters for catboost");
}

try
{
    if (!catboost.CalcModelPrediction(
                modelHandle,
                1,
                ppFloatFeatures, 10,
                ppCatFeatures, 4,
                pResult, 1
                ))
    {
        throw new RuntimeException("No prediction made: " + catboost.GetErrorString());
    }
    else
    {
        double[] result = pResult.getDoubleArray(0, 1);
        log.info("Catboost prediction: " + String.valueOf(result[0]));
        Assert.assertFalse("ERROR: Result empty", result.length == 0);
    }
}
catch (Exception e)
{
    throw new RuntimeException("Prediction failed: " + e);
}

I've tried passing Pointer, PointerByReference and Pointer[] to the CalcModelPrediction function in place of float **floatFeatures and char ***catFeatures but nothing worked. I always get a segmentation fault, presumably when the CalcModelPrediction function attempts to get elements of floatFeatures and catFeatures by calling floatFeatures[0][0] and catFeatures[0][0].

So the question is, what's the right way of passing a multi-dimensional array from Java through JNA into C, where it could be treated as a pointer to a pointer to a value?

An interesting thing is that the CalcModelPredictionFlat function that accepts only float **floatFeatures and then simply calls *floatFeatures, works perfectly fine when passing PointerByReference.

UPDATE - 5.5.2018

Part 1

After trying to debug the segfault by slightly modifying the original Catboost .cpp and .h files and recompiling the libcatboost.so library, I found out that the segfault was due to me mapping size_t in C to int in Java. After fixing this, my interface function in Java looks like this:

public interface CatboostModel extends Library {
        public boolean LoadFullModelFromFile(Pointer calcer, String filename);
        public boolean CalcModelPrediction(Pointer calcer, size_t docCount,
                Pointer[] floatFeatures, size_t floatFeaturesSize,
                String[] catFeatures, size_t catFeaturesSize,
                Pointer result, size_t resultSize);
    }

Where the size_t Class is defined as follows:

public static class size_t extends IntegerType {                                           
    public size_t() { this(0); }                                                           
    public size_t(long value) { super(Native.SIZE_T_SIZE, value); }                       
}

Part 2 Looking more into the Catboost code, I've noticed that **floatFeatures are being accessed by rows, like floatFeatures[i] while ***catFeatures are accessed by rows and columns, like catFetures[i][catFeatureIdx].

After changing floatFeatures in Java to an array of Pointer, my code started to work with the model trained without categorical features, i.e. catFeatures length is zero.

This trick, however, didn't work with catFeatures that are accessed through a double subscript operator [i][catFeatureidx]. So for now, I modified the original Catboost code so that it would accept char **catFeatures - an array of strings. In Java interface function, I set String[] catFeatures. Now I can make predictions for one element at a time, which is not ideal.

Dmitri Lihhatsov

I've managed to make it all work with the original Catboost code and libcatboost.so.

The Java interface function is defined like this. Note that in order to emulate a 2D array (or a pointer to a pointer) of float values and strings, I'm using Pointer[] type.

public interface CatboostModel extends Library {
        public boolean LoadFullModelFromFile(Pointer calcer, String filename);
        public boolean CalcModelPrediction(Pointer calcer, size_t docCount,
                Pointer[] floatFeatures, size_t floatFeaturesSize,
                Pointer[] catFeatures, size_t catFeaturesSize,
                Pointer result, size_t resultSize);
    }

After that, I populate the floatFeatures and catFeatures parameters like this (some dummy data here). Note that for strings I'm using JNA's StringArray.

float[] floatFeatures = {0.4f, 0.8f, 0.3f, 0.3f, 0.1f, 0.5f, 0.4f, 0.8f, 0.3f, 0.3f} ;
String[] catFeatures = {"1", "2", "3", "4"};

final Pointer pFloatFeatures = new Memory(floatFeatures.length * Native.getNativeSize(Float.TYPE));
final Pointer[] ppFloatFeatures = new Pointer[2];
for (int dloop=0; dloop<10; dloop++) {
    pFloatFeatures.setFloat(dloop * Native.getNativeSize(Float.TYPE), floatFeatures[dloop]);
}
ppFloatFeatures[0] = pFloatFeatures;
ppFloatFeatures[1] = pFloatFeatures;

final Pointer[] ppCatFeatures = new Pointer[catFeatures.length];
final Pointer pCatFeatures = new StringArray(catFeatures);
ppCatFeatures[0] = pCatFeatures;
ppCatFeatures[1] = pCatFeatures;

Finally, I pass these parameters to Catboost:

if (!catboost.CalcModelPrediction(
                modelHandle,
                new size_t(2L),
                ppFloatFeatures, new size_t((long)floatFeatures.length),
                ppCatFeatures, new size_t((long)catFeatures.length),
                pResult, new size_t(2L)
                ))
{
    throw new RuntimeException("No prediction made: " + catboost.GetErrorString());
}

To get predictions we can do:

double[] result = pResult.getDoubleArray(0, 2);

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Java

JNA passing the equivalent of swift pointer from Java Android to C

From Java

JNA : Get value from a pointer to a pointer to a structure

From Java

What is the difference between passing by reference in Java and passing a pointer in C?

From Java

Pass InputStream through JNA to C code as a File Pointer

From Java

Passing a pointer from JNI to Java using a long

From Java

Passing pointer from java to native

From

Passing pointer from go to c-dll

From Java

JNA passing structure containing pointer to structure(s) and pointer to primitive

From Dev

Properly passing pointer from function C++

From Java

Pass pointer or PointerByReference in JNA for pointer argument in C

From Dev

Passing pointer by reference in C

From Dev

Confusion with weird C pointer behavior when passing through functions

From Dev

C++: Syntax of passing a pointer array through a function

From Dev

Passing a pointer to a pointer vs passing a pointer for initialisation in C

From Dev

Passing pointer of dynamic var by reference

From Dev

Passing function pointer from native to java

From Dev

C passing dynamic array through pointer (Segmentation Fault)

From Dev

Passing pointer from C# to C++

From Dev

Pointer issues when mapping native C functions to Java interface with JNA

From Dev

passing a pointer through a function c

From Dev

passing a pointer address vs pointer to a pointer in C

From Dev

C - Passing pointer to function

From Dev

C++ Iterating through map from pointer

From Dev

Passing pointer of a pointer in C

From Dev

Why is passing java long as pointer to struct in c++ library crashing app?

From Dev

C++: Passing lambda pointer as a function pointer

From Dev

How to call C function through JNA with 2d pointer?

From Dev

Mallocing and Freeing in C, but passing the pointer through Python via ctypes

From Dev

Is passing pointer to an array as pointer to pointer UB in C?

Related Related

  1. 1

    JNA passing the equivalent of swift pointer from Java Android to C

  2. 2

    JNA : Get value from a pointer to a pointer to a structure

  3. 3

    What is the difference between passing by reference in Java and passing a pointer in C?

  4. 4

    Pass InputStream through JNA to C code as a File Pointer

  5. 5

    Passing a pointer from JNI to Java using a long

  6. 6

    Passing pointer from java to native

  7. 7

    Passing pointer from go to c-dll

  8. 8

    JNA passing structure containing pointer to structure(s) and pointer to primitive

  9. 9

    Properly passing pointer from function C++

  10. 10

    Pass pointer or PointerByReference in JNA for pointer argument in C

  11. 11

    Passing pointer by reference in C

  12. 12

    Confusion with weird C pointer behavior when passing through functions

  13. 13

    C++: Syntax of passing a pointer array through a function

  14. 14

    Passing a pointer to a pointer vs passing a pointer for initialisation in C

  15. 15

    Passing pointer of dynamic var by reference

  16. 16

    Passing function pointer from native to java

  17. 17

    C passing dynamic array through pointer (Segmentation Fault)

  18. 18

    Passing pointer from C# to C++

  19. 19

    Pointer issues when mapping native C functions to Java interface with JNA

  20. 20

    passing a pointer through a function c

  21. 21

    passing a pointer address vs pointer to a pointer in C

  22. 22

    C - Passing pointer to function

  23. 23

    C++ Iterating through map from pointer

  24. 24

    Passing pointer of a pointer in C

  25. 25

    Why is passing java long as pointer to struct in c++ library crashing app?

  26. 26

    C++: Passing lambda pointer as a function pointer

  27. 27

    How to call C function through JNA with 2d pointer?

  28. 28

    Mallocing and Freeing in C, but passing the pointer through Python via ctypes

  29. 29

    Is passing pointer to an array as pointer to pointer UB in C?

HotTag

Archive