Input

The input to Shroud is a YAML formatted file. YAML is a human friendly data serialization standard. [yaml] Structure is shown through indentation (one or more spaces). Sequence items are denoted by a dash, and key value pairs within a map are separated by a colon:

library: Tutorial

declarations:
- decl: typedef int TypeID

- decl: void Function1()

- decl: class Class1
  declarations:
  - decl: void Method1()

Each decl entry corresponds to a line of C or C++ code. The top level declarations field represents the source file while nested declarations fields corresponds to curly brace blocks. The above YAML file represent the source file:

typedef int TypeID;

void Function1();

class Class1
{
    void Method1();
}

A block can be used to group a collection of decl entires. Any option or format fields will apply to all declarations in the group:

declarations:
- block: True
  options:
    F_name_impl_template: {library}_{F_name_api}
  format:
    F_impl_filename: localfile.f
  declarations:
  - decl: void func1()
  - decl: void func2()

Shroud use curly braces for format strings. If a string starts with a curly brace YAML will interpret it as a map/dictionary instead of as part of the string. To avoid this behavior, strings which start with a curly brace should be quoted:

name : "{fmt}"

Strings may be split across several lines by indenting the continued line:

- decl: void Sum(int len, const int *values+rank(1),
                 int *result+intent(out))

Some values consist of blocks of code. The pipe, |, is used to indicate that the string will span several lines and that newlines should be preserved:

C_invalid_name: |
    if (! isNameValid({cxx_var})) {{
        return NULL;
    }}

Note that to insert a literal {, a double brace, {{, is required since single braces are used for variable expansion. {cxx_var} in this example. However, using the pipe, it is not necessary to quote lines that contain other YAML meta characters such as colon and curly braces.

For example, YAML will get confused by the :: characters and try to create a dictionary with the key integer, parameter :.

splicer_code:
  f:
    module_top:
    - integer, parameter :: INDEXTYPE = 5

Literal newlines, /n, are respected. Line lengths are controlled by the options C_line_length and F_line_length and default to 72.:

C_invalid_name: |
    if (! isNameValid({cxx_var})) {{+
    return NULL;
    -}}

The only formatting option is to control output line lengths. This is required for Fortran which has a maximum line length of 132 in free form which is generated by shroud. If you care where curly braces go in the C source then it is best to set C_line_length to a large number then use an external formatting tool such as indent or uncrustify.

Customizing Behavior in the YAML file

Fields

A field only applies to the type, enumeration, function, structure or class to which it belongs. It is not inherited. For example, cxx_header is a field which is used to define the header file for class Names. Likewise, setting library within a class does not change the library name.

library: testnames

declarations:
  - decl: class Names
    cxx_header: names.hpp
    declarations:
    -  decl: void method1

Options

Options are used to customize the behavior of Shroud. They are defined in the YAML file as a dictionary. Options can be defined at the global, class, or function level. Each level creates a new scope which can access all upper level options. This allows the user to modify behavior for all functions or just a single one:

options:
  option_a = false
  option_b = false
  option_c = false

declarations:
- class: class1
  options:
#    option_a = false     # inherited
     option_b = true
#    option_c = false     # inherited
  declarations:
  - decl: void function1
    options:
#     option_a = false    # inherited
#     option_b = true     # inherited
      option_c = true

Format

A format dictionary contains strings which can be inserted into generated code. Generated filenames are also entries in the format dictionary. Format dictionaries are also scoped like options. For example, setting a format in a class also effects all of the functions within the class.

How code is formatted

Format strings contain “replacement fields” surrounded by curly braces {}. Anything that is not contained in braces is considered literal text, which is copied unchanged to the output. If you need to include a brace character in the literal text, it can be escaped by doubling: {{ and }}. [Python_Format]

There are some metacharacters that are used for formatting the line:

\f

Add an explicit formfeed

\t

A tab is used to suggest a place to break the line for a continuation before it exceeds option C_line_length or F_line_length. Any whitespace after a tab will be trimmed if the line is actually split at the tab. If a continuation was not needed (there was enough space on the current line) then the tab has no effect:

arg1,\t arg2

+ -

Increase or decrease indention indention level. Used at the beginning or end of a line:

if (condition) {{+
do_one();
-}} else {{+
do_two();
-}}

The double curly braces are replace by a single curly. This will be indented as:

if (condition) {
    do_one();
} else {
    do_two();
}

#

If the first character is a #, ignore indention and write in column 0. Useful for preprocessing directives.

^

If the first character is ^, ignore indention and write in column 0. Useful for comments or labels.

@

If the first character is @, treat the following character literally. Used to ignore a metacharacter:

struct aa = {{++
0// set field to 0
@0,
-}};

Formatted as:

struct aa = {
// set field to 0
    0,
};

Attributes

Annotations or attributes apply to specific arguments or results. They describe semantic behavior for an argument. An attribute may be set to true by listing its name or it may have a value in parens:

- decl: Class1()  +name(new)
- decl: void Sum(int len, const int *values+rank(1)+intent(in))
- decl: const std::string getName() +len(30)

Attributes may also be added external to decl:

- decl: void Sum(int len, const int *values)
  attrs:
      values:
          intent: in
          rank: 1
- decl: const std::string getName()
  fattrs:
      len: 30

Attributes must be added before default arguments since a default argument may include a plus symbol:

- decl: void Sum(int len, const int *values+rank(1)+intent(in) =nullptr)

api

Controls the API used by the C wrapper. Normally, this attribute is determined by Shroud internally based on the argument type and attributes. But in some cases it’s helpful if the user defines it.

buf

Pass a pointer to a string along with meta data about the length or size. Pointers to native and char use additional metadata extracted by the Fortran wrapper via intrinsics LEN and SIZE. In addition, intent(in) strings will be copied and null-terminated.

capi

There is currently one useful case where the user would want to set this attribute. To avoid creating a wrapper which copies and null terminates a char * argument the user can set api(capi). The address of the formal parameter will be passed to the user’s library. This is useful when null termination does not make sense. For example, when the argument is a large buffer to be written to a file. The C library must have some other way of determining the length of the argument such as another argument with the explicit length.

capptr capsule

The capsule and capptr APIs are used by the capsule created by shadow types created for C++ classes. In both cases the result is passed from Fortran to C as an extra argument for function which return a class. With capptr, the C wrapper will return a pointer to the capsule argument while capsule will not return a value for the function. This is controlled by the C_shadow_result option.

cdesc

Uses a struct to pass the metadata between Fortran and C. This struct contains additional metadata beyond the length and size used by api(buf). It is also used to return metadata from the C wrapper to the Fortran wrapper. The metadata includes shape information from a dimension attribute and is used on intent(OUT) arguments and function results to set the shape of Fortran POINTER variables. The struct is named by the format fields C_array_type and F_array_type. It is similar to the CFI_cdesc_t struct provied by Fortran 2018.

cfi

Use Further interoperability with C features and pass CFI_cdesc_t arguments to the C wrapper which will extract the metadata. Using option F_CFI will make this the default behavior.

this

Used when the return_this field is True. The C++ function returns a pointer to the this variable.

assumedtype

When this attribute is applied to a void * argument, the Fortran assumed-type declaration, type(*), will be used. Since Fortran defaults to pass-by-reference, the argument will be passed to C as a void * argument. The C function will need some other mechanism to determine the type of the argument before dereferencing the pointer. Note that assumed-type is part of Fortran 2018.

blanknull

Used with const char * arguments to convert a blank string to a NULL pointer instead of an empty C string ('\0'). Can be applied to all arguments with the option F_blanknull.

charlen

charlen is used to define the size of a char *arg+intent(out) argument in the Python wrapper. This deals with the case where arg is provided by the user and the function writes into the provided space. This technique has the inherent risk of overwritting memory if the supplied buffer is not long enough. For example, when used in C the user would write:

#define API_CHARLEN
char buffer[API_CHARLEN];
fill_buffer(buffer);

The Python wrapper must know the assumed length before calling the function. It will then be converted into a str object by PyString_FromString.

Fortran does not use this attribute since the buffer argument is supplied by the user. However, it is useful to provide the parameter by adding a splicer block in the YAML file:

splicer_code:
  f:
    module_top:
    -  "integer, parameter :: MAXNAME = 20"

Warning

Using charlen and dimension together is not currently supported.

constfunc

Add to a function decl when C++ const cannot be used, perhaps to help distinguish overloaded functions. This will ensure that the shadow argument is intent(in) instead of defaulting to intent(INOUT). This allows the function to be used with intent(IN) subprogram dummy argument.

custom

A user defined string which is used when creating the statement group name. This allows some custom behavior in the wrapper. For example, by defining the c_call field with other code. This is used by ‘std::weak_ptr`` to perform assignment on the pointer which is not part of the C++ library being wrapped.

default

Default value for C++ function argument. This value is implied by C++ default argument syntax.

deref

Define how to dereference function results and pointers which are returned via an argument. It’s also used with objects which represent an array such as std::string or std::vector. This may be used in conjunction with dimension to create arrays. For example, int **out +intent(out)+deref(pointer)+dimension(10).

allocatable

For Fortran, add ALLOCATABLE attribute to argument. An ALLOCATE statement is added and the contents of the C++ argument is copied. If owner(caller) is also defined, the C++ argument is released. The caller is responsible to DEALLOCATE the array.

For Python, create a NumPy array (same as pointer attribute)

copy

Copy results into the Fortran argument. This helps reduce memory management problems since there is no dynamic memory. In addition, this helps with non-contiguous C++ memory such as arrays or vectors of char * or std::string. Fortran can not deal with ragged arrays directly and will copy into the contiguous argument.

pointer

For intent(in) arguments, a POINTER Fortran attribute will be added. This allows a dynamic memory address to be passs to the library.

void giveMemory(arg *data +intent(in)+deref(pointer))

For intent(out) arguments this indicates that memory from the library is being passed back to the user and will be assigned using c_f_pointer.

Attribute owner(caller) will be set to option.default_owner if it is not also defined. If caller an additional argument is added which is used to release the memory.

For Python, create a list or NumPy array.

- decl: double *ReturnPtrFun() +dimension(10)
- decl: void ReturnPtrArg(double **arg +intent(out)+dimension(10))

- decl: double *ReturnScalar() +deref(pointer)

A pointer to scalar will also return a NumPy array in Python. Use +deref(scalar) to get a scalar.

raw

Treat the pointer as a void *. Required when there are several layers of indirection that do not map directly to the wrapper language.

Attribute owner(caller) will be set to option.default_owner if it is not also defined.

For Fortran, return a type(C_PTR).

For Python, return a PyCapsule.

scalar

Treat the pointee as a scalar. For Fortran, return a scalar and not a pointer to the scalar. For Python, this will not create a NumPy object.

destructor_name

Specifies a name in the destructors section which lists code to be used to release memory. Used with function results. It is used in the C_memory_dtor_function and will have the variable void *ptr available as the pointer to the memory to be released. See Memory Management for details.

declarations:
- decl: char *getName() +destructor_name(free_getName)

destructors:
   free_getName: |
      decref(ptr);

dimension

A list of array extents for pointer or reference variables. All arrays use the language’s default lower-bound (1 for Fortran and 0 for Python). Used to define the dimension of pointer arguments with intent(out) and function results. It can also be used with class member variables to create a getter which returns a Fortran pointer. A dimension without any value is an error – +dimension.

The expression is evaluated in the C wrapper. It can be passed back to the Fortran wrapper via a cdesc argument of type F_array_type when the attribute deref is set to allocatable or pointer. This allows the shape to be used in an ALLOCATE statement or a call to C_F_POINTER.

For Futher interoperability with C, set with option F_CFI, the shape is used directly in the C wrapper in a call to CFI_allocate or CFI_establish.

struct {
  int len;
  double *array +dimension(len);
};

An expression can also contain a intent(out) argument of the function being wrapped.

int * get_array(int **count +intent(out)+hidden) +dimension(count)

Argument count will be used to define the shape of the function result but will not be part of the wrapped API since it is hidden.

rank and dimension can not be specified together.

The dimension may also be assumed-rank, dimension(..), to allow scalar or any rank. If option F_CFI is true, then assumed-rank will be added to the function interface and the C wrapper will extract the rank from the CFI_cdesc_t argument. Otherwise, a generic function will be created for each rank requested by options F_assumed_rank_min and “F_assumed_rank_max.

external

This attribute is only valid with function pointers. It will ensure that a Fortran wrapper is created which uses the external statement for the argument. This will allow any function to be used as the dummy argument for the function pointer.

See also the funptr attribute. See Function Pointers.

funcarg

Used with functions to convert it to a subroutine and to treat the result as an additional argument. This is helpful with functions that return a pointer. Instead, the Fortran wrapper accepts an argument from the user which will contain a copy of the result. The result may also use the deref attribute to create an ALLOCATABLE or POINTER result similar to an intent(out) argument.

The attribute value can contain the name for the argument. If no name is given then the option F_result_as_arg will be used to name the argument. The attribute value is assigned to format fields f_var and i_var.

funptr

This attribute is only valid with function pointers. Create a type(C_FUNPTR) argument for the function pointer instead of an abstract interface. The caller is required to use C_FUNLOC to get the address of the function. The advantage is that this will allow any function to be passed. Like void pointers, the user is responsible to ensure the function pointer is called with the correct arguments.

See also the external attribute. See Function Pointers.

hidden

The argument will not appear in the Fortran or Python API.

For the native C API it will appear as a regular argument. For the bufferify C API, it will be a local variable which is passed to the C++ function.

It is useful for a function which returns the length of another pointer arguments. This value is save in the F_array_type argument or the CFI_cdesc_t struct.

For example, setting the shape of a pointer function:

   int * ReturnIntPtr(int *len+intent(out)+hidden) +dimension(len)+deref(pointer)

Will create a Fortran wrapper which returns a ``POINTER`` which
is ``len`` long but does not have an argument for the length.

integer(C_INT), pointer :: rv(:)
rv = return_int_ptr()
! size(rv)  is argument len

The api attribute will be set to hidden before looking up the Fortran statement group.

implied

The value of an arguments to the C++ function may be implied by other arguments. If so the implied attribute can be used to assign the value to the argument and it will not be included in the wrapped API.

Used to compute value of argument to C++ based on argument to Fortran or Python wrapper. Useful with array sizes:

int Sum(const int * array, int len +implied(size(array))

Several functions will be converted to the corresponding code for Python wrappers: size, len and len_trim.

• size(array[,dim]) Determine the extent of array along a specified dimension dim, or the total number of elements in array if dim is absent.

  • array name of argument

  • dim rank of array to check. If none, entire array.

• len(string) Returns the length of a character string.

• len_trim(string) Returns the length of a character string, ignoring any trailing blanks.

intent

The Fortran intent of the argument. Valid values are in, out, inout.

in

The argument will only be read from.

inout

The argument will be read from and written to.

out

The argument will be written to.

none

No intent. Default for function pointer arguments.

Nonpointer arguments can only be intent(in). If the argument is const, the default is in.

In Python, intent(out) arguments are not used as input arguments to the function but are returned as values.

Internally, Shroud also assigns the values of function, operator, ctor and dtor.

len

When used with a function, it will be the length of the return value of the function using the declaration:

character(kind=C_CHAR, len={c_var_len}) :: {F_result}

name

Name of the method. Useful for constructor and destructor methods which have default names ctor and dtor. Also useful when class member variables use a convention such as m_variable. The name can be set to variable to avoid polluting the Fortran interface with the m_ prefix. Fortran and Python both have an explicit scope of self%variable and self.variable instead of an implied this.

operator

Defines the function as a Fortran operator. Used to create assignment overloads.

owner

Specifies who is responsible to release the memory associated with the argument/result.

The terms follow Python’s reference counting . [Python_Refcount] The default is set by option default_owner which is initialized to borrow.

caller

The memory belongs to the user who is responsible to delete it. A shadow class must have a destructor wrapped in order to delete the memory.

library

The memory belongs to the library and should not be deleted by the user. This is the default value.

pass

Used to define the argument which is the passed-object dummy argument for type-bound procedures when treating a struct as a class. In C, which does not support the class keyword, a struct can be used as a class by defining option wrap_struct_as=class. Other functions can be associated with the class by setting option class_method to the name of the struct.

See detail at Object-oriented C

rank

Add an assumed-shape dimension with the given rank. rank must be 0-7. A rank of 0 implies a scalar argument.

double *array +rank(2)

Creates the declaration:

real(C_DOUBLE) :: array(:,:)

Use with +intent(in) arguments when the wrapper should accept any extent instead of using Fortran’s assumed-shape with dimension(:).

This can be simpler than the dimension attribute for multidimension arrays. rank and dimension can not be specified together.

For the bind(C) interface, an assumed-size array will be created for any array with rank > 0.

real(C_DOUBLE) :: array(*)

readonly

May be added to struct or class member to avoid creating a setter function. If the member is const, this attribute is added by Shroud.

value

If true, pass-by-value; else, pass-by-reference. This attribute is implied when the argument is not a pointer or reference. This will also default to intent(IN) since there is no way to return a value.

Note

The Fortran wrapper may use an intrinsic function for some attributes. For example, len, len_trim, and size. If there is an argument with the same name, the generated code may not compile.

Shroud preserves the names of the arguments since Fortran allows them to be used in function calls - call worker(len=10)

Statements

The code generated for each argument and return value can be controlled by statement dictionaries. Shroud has many entries built in which are used for most arguments. But it is possible to add custom code to the wrapper by providing additional fields. Most wrappers will not need to provide this information.

An example from strings.yaml:

- decl: const string * getConstStringPtrLen() +len=30
  doxygen:
    brief: return a 'const string *' as character(30)
    description: |
      It is the caller's responsibility to release the string
      created by the C++ library.
      This is accomplished with C_finalize_buf which is possible
      because +len(30) so the contents are copied before returning.
  fstatements:
    f:
      c_final:
      - delete {cxx_var};

An example from vectors.yaml:

- decl: void vector_iota_out_with_num(std::vector<int> &arg+intent(out))
  fstatements:
    c:
      c_return_type: long
      c_return:
      - return Darg->size;
    f:
      c_return_type: long
      c_return:
      - return SHT_arg_cdesc->size;
      f_result: num
      f_dummy_decl:
      -  "integer(C_LONG) :: {F_result}"
      f_call:
      -  "{F_result} = {F_C_call}({F_arg_c_call})"
      f_module:
        iso_c_binding: ["C_LONG"]

A complete description is in the Statements chapter.

Patterns

To address the issue of semantic differences between Fortran and C++, patterns may be used to insert additional code. A pattern is a code template which is inserted at a specific point in the wrapper. They are defined in the input YAML file:

declarations:
- decl: const string& getString2+len=30()
  C_error_pattern: C_invalid_name

patterns:
  C_invalid_name: |
      if ({cxx_var}.empty()) {{
          return NULL;
      }}

The C_error_pattern will insert code after the call to the C++ function in the C wrapper and before any post_call sections from the types. The bufferified version of a function will append _buf to the C_error_pattern value. The pattern is formatted using the context of the return argument if present, otherwise the context of the function is used. This means that c_var and c_var_len refer to the argument which is added to contain the function result for the _buf pattern.

The function getString2 is returning a std::string reference. Since C and Fortran cannot deal with this directly, the empty string is converted into a NULL pointer:: will blank fill the result:

const char * STR_get_string2()
{
    const std::string & SHCXX_rv = getString2();
    // C_error_pattern
    if (SHCXX_rv.empty()) {
        return NULL;
    }
    const char * SHC_rv = SHCXX_rv.c_str();
    return SHC_rv;
}

Splicers

No matter how many features are added to Shroud there will always exist cases that it does not handle. One of the weaknesses of generated code is that if the generated code is edited it becomes difficult to regenerate the code and preserve the edits. To deal with this situation each block of generated code is surrounded by ‘splicer’ comments:

const char * STR_get_char3()
{
    // splicer begin function.get_char3
    const char * SH_rv = getChar3();
    return SH_rv;
    // splicer end function.get_char3
}

These comments delineate a section of code which can be replaced by the user. The splicer’s name, function.get_char3 in the example, is used to determine where to insert the code.

There are two ways to define splicers in the YAML file. First add a list of files which contain the splicer text:

splicer:
  f:
  -  fsplicer.f
  c:
  -  csplicer.c

In the listed file, add the begin and end splicer comments, then add the code which should be inserted into the wrapper inbetween the comments. Multiple splicer can be added to an input file. Any text that is not within a splicer block is ignored. Splicers must be sorted by language. If the input file ends with .f or .f90 it is processed as splicers for the generated Fortran code. Code for the C wrappers must end with any of .c, .h, .cpp, .hpp, .cxx, .hxx, .cc, .C:

-- Lines outside blocks are ignore
// splicer begin function.get_char3
const char * SH_rv = getChar3();
SH_rv[0] = 'F';    // replace first character for Fortran
return SH_rv + 1;
// splicer end function.get_char3

This technique is useful when the splicers are very large or are generated by some other process.

The second method is to add the splicer code directly into the YAML file. A splicer can be added after the decl line. This splicer takes priority over other ways of defining splicers.

- decl: bool isNameValid(const std::string& name)
  splicer:
     c:
     - "return name != NULL;"
     c_buf:
     - // Added to the bufferify C wrapper called by Fortran
     f:
     - 'rv = name .ne. " "'

A splicer can be added in the splicer_code section. This can be used to add code to spliers which do not correspond directly to a declaration. Each level of splicer is a mapping and each line of text is an array entry:

splicer_code:
  c:
    function:
      get_char3:
      - const char * SH_rv = getChar3();
      - SH_rv[0] = 'F';    // replace first character for Fortran
      - return SH_rv + 1;

In addition to replacing code for a function wrapper, there are splicers that are generated which allow a user to insert additional code for helper functions or declarations:

! file_top
module {F_module_name}
   ! module_use
   implicit none
   ! module_top

   type class1
     ! class.{cxx_class}.component_part
   contains
     ! class.{cxx_class}.generic.{F_name_generic}
     ! class.{cxx_class}.type_bound_procedure_part
   end type class1

   ! additional_declarations

   contains

   ! function.{F_name_function}

   ! {cxx_class}.method.{F_name_function}

   ! additional_functions

end module {F_module_name}

C header:

// class.{class_name}.CXX_declarations

extern "C" {
// class.{class_name}.C_declarations
}

C implementation:

// class.{class_name}.CXX_definitions

extern "C" {
  // class.{class_name}.C_definitions

  // function.{C_name_api}{function_suffix}

  // class.{cxx_class}.method.{C_name_api}{function_suffix}

}

The splicer comments can be eliminated by setting the option show_splicer_comments to false. This may be useful to eliminate the clutter of the splicer comments.

statements

Statements add control over how specific function and argument wrapping is controlled based on language, intent, and attributes. This section is not needed for causual use of Shroud since the defaults are sufficient for most cases.

statements:
  fc:
    file: [ ]
    extend:
    - name: f_...

See Statements for details.

file_code

The file_code section allows the user to add some additional code to the wrapper which may conflict with code automatically added by Shroud for typemaps, statements or helpers. While splicer are simple text insertation, file_code inserts code semantically.

For C wrappers, including header files may duplicate headers added when creating the wrapper. By listing them in a file_code section instead of a splicer Shroud is able to manage all header files.

For Fortran wrappers, USE statements are managed collectively to avoid redundant USE statements.

file_code:
  wraptypemap.h:
    c_header: <stdint.h>
    cxx_header: <cstdint>
  wrapftypemap.f:
    f_module:
      iso_c_binding:
      - C_INT32_T
      - C_INT64_T

Footnotes

[Python_Format]

https://docs.python.org/2/library/string.html#format-string-syntax

[Python_Refcount]

https://docs.python.org/3/c-api/intro.html#reference-count-details

[yaml]

yaml.org