*
This commit is contained in:
@@ -0,0 +1,4 @@
|
||||
__pycache__
|
||||
.Trash*
|
||||
/build
|
||||
/.tup
|
||||
@@ -0,0 +1,127 @@
|
||||
## Overview ##
|
||||
|
||||
The goal of Fibre is to provide a framework to make suckless distributed applications easier to program.
|
||||
|
||||
In particular:
|
||||
|
||||
- Nobody likes boiler plate code. Using a remote object should feel almost
|
||||
exactly as if it was local. No matter if it's in a different process, on
|
||||
a USB device, connected via Bluetooth, over the Internet or all at the
|
||||
same time.
|
||||
All complexity arising from the system being distributed should be taken
|
||||
care of by Fibre while still allowing the application developer to easily
|
||||
fine-tune things.
|
||||
|
||||
- Fibre has the ambition to run on most major platforms and provide bindings
|
||||
for the most popular languages. Even bare metal embedded systems with very
|
||||
limited resources. See the Compatibility section for the current status.
|
||||
|
||||
- Once you deployed your application and want to change the interface, don't
|
||||
worry about breaking other applications. With Fibre's object model, the
|
||||
most common updates like adding methods, properties or arguments won't
|
||||
break anything. Sometimes you can get away with removing methods if they
|
||||
weren't used by other programs. **This is not implemented yet.**
|
||||
|
||||
## Current Status ##
|
||||
|
||||
The project is in an early stage and the focus so far was to get a minimum working implementation.
|
||||
|
||||
* **C++**: Currently only supports the server side (i.e. publishing local objects). The C++ library comes with builtin support for TCP and UDP transport layers on Posix platforms. The library can easily be used with user provided transport layers.
|
||||
|
||||
* **Python**: Currently only supports the client side (i.e. using remote objects). The Python library comes with builtin support for TCP, UDP, USB and UART transport layers.
|
||||
|
||||
Support for more languages (most importantly JavaScript) will be added once the protocol matures. Feel free to add your contribution.
|
||||
|
||||
## Show me some code
|
||||
|
||||
Consider this program:
|
||||
|
||||
```
|
||||
class TestClass {
|
||||
public:
|
||||
float property1;
|
||||
float property2;
|
||||
|
||||
float set_both(float arg1, float arg2) {
|
||||
property1 = arg1;
|
||||
property2 = arg2;
|
||||
return property1 + property2;
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
int main() {
|
||||
TestClass test_object = TestClass();
|
||||
|
||||
while (1) {
|
||||
printf("test_object.property1: %f\n", test_object.property1);
|
||||
usleep(1000000 / 5); // 5 Hz
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Say you want to publish `test_object` so that a remote Fibre node can use it.
|
||||
|
||||
1. Add includes
|
||||
```C++
|
||||
#include <fibre/protocol.hpp>
|
||||
#include <fibre/posix_tcp.hpp>
|
||||
```
|
||||
1. Add Fibre export definitions to the exported class
|
||||
```C++
|
||||
class TestClass {
|
||||
[...]
|
||||
FIBRE_EXPORTS(TestClass,
|
||||
make_protocol_property("property1", &property1),
|
||||
make_protocol_property("property2", &property2),
|
||||
make_protocol_function("set_both", *obj, &TestClass::set_both, "arg1", "arg2")
|
||||
);
|
||||
};
|
||||
```
|
||||
Note: in the future this will be generated from a YAML file using automatic code generation.
|
||||
|
||||
1. Publish the object on Fibre
|
||||
```C++
|
||||
auto definitions = test_object.fibre_definitions;
|
||||
fibre_publish(definitions);
|
||||
```
|
||||
Note: currently you must publish all objects at once. This will be fixed in the future.
|
||||
|
||||
1. Start the TCP server
|
||||
```C++
|
||||
std::thread server_thread_tcp(serve_on_tcp, 9910);
|
||||
```
|
||||
Note: this step will be replaced by a simple `fibre_start()` call in the future. All builtin transport layers then will be started automatically.
|
||||
|
||||
## Adding Fibre to your project ##
|
||||
|
||||
We recommend Git subtrees if you want to include the Fibre source code in another project.
|
||||
Other contributors don't need to know anything about subtrees, to them the Fibre repo will be like any other normal directory.
|
||||
|
||||
#### Adding the repo
|
||||
```
|
||||
git remote add fibre-origin git@github.com:samuelsadok/fibre.git
|
||||
git fetch fibre-origin
|
||||
git subtree add --prefix=fibre --squash fibre-origin master
|
||||
```
|
||||
Instead of using the upstream remote, you might want to use your own fork for greater flexibility.
|
||||
|
||||
#### Pulling updates from upstream
|
||||
```
|
||||
git subtree pull --prefix=fibre --squash fibre-origin master
|
||||
```
|
||||
|
||||
#### Contributing changes back to upstream
|
||||
This requires push access to `fibre-origin`.
|
||||
```
|
||||
git subtree push --prefix=fibre fibre-origin master
|
||||
```
|
||||
|
||||
## Projects using Fibre ##
|
||||
|
||||
- [ODrive](https://github.com/madcowswe/ODrive): High performance motor control
|
||||
- [lightd](https://github.com/samuelsadok/lightd): Service that can be run on a Raspberry Pi (or similar) to control RGB LED strips
|
||||
|
||||
## Contribute ##
|
||||
|
||||
This project losely adheres to the [Google C++ Style Guide](https://google.github.io/styleguide/cppguide.html).
|
||||
@@ -0,0 +1,90 @@
|
||||
/*[# This is the original template, thus the warning below does not apply to this file #]
|
||||
* ============================ WARNING ============================
|
||||
* ==== This is an autogenerated file. ====
|
||||
* ==== Any changes to this file will be lost when recompiling. ====
|
||||
* =================================================================
|
||||
*
|
||||
* This file contains the toplevel handler for Fibre v0.1 endpoint operations.
|
||||
*
|
||||
* This endpoint-oriented approach will be deprecated in Fibre v0.2 in favor of
|
||||
* a function-oriented approach and a more powerful object model.
|
||||
*
|
||||
*/
|
||||
#ifndef __FIBRE_INTERFACES_HPP
|
||||
#define __FIBRE_INTERFACES_HPP
|
||||
|
||||
#include <fibre/introspection.hpp>
|
||||
|
||||
// Note: with -Og the functions with large switch statements reserves a huge amount
|
||||
// of stack space because they reserves separate space for the stack frame of each
|
||||
// of the inlined functions.
|
||||
// The minimum known set of flags to prevent this is `-O1 -fipa-sra`.
|
||||
// `-O2`, `-O3` and `-Os` are supersets of this.
|
||||
|
||||
#pragma GCC push_options
|
||||
#pragma GCC optimize ("s")
|
||||
|
||||
namespace fibre {
|
||||
|
||||
const unsigned char embedded_json[] = [[embedded_endpoint_definitions | to_c_string]];
|
||||
const size_t embedded_json_length = sizeof(embedded_json) - 1;
|
||||
const uint16_t json_crc_ = calc_crc16<CANONICAL_CRC16_POLYNOMIAL>(PROTOCOL_VERSION, embedded_json, embedded_json_length);
|
||||
const uint32_t json_version_id_ = (json_crc_ << 16) | calc_crc16<CANONICAL_CRC16_POLYNOMIAL>(json_crc_, embedded_json, embedded_json_length);
|
||||
|
||||
static void get_property(Introspectable& result, size_t idx) {
|
||||
switch (idx) {
|
||||
[%- for endpoint in endpoints %]
|
||||
[%- if endpoint.function.name == 'exchange' and endpoint.in_bindings | list == ['obj'] %]
|
||||
case [[endpoint.id]]: { [[(endpoint.in_bindings['obj'] + '$') | replace(')$', ', &result.storage_)')]]; result.type_info_ = &FibrePropertyTypeInfo<[[endpoint.function.in['obj'].type.c_name]]>::singleton; } break;
|
||||
[%- endif %]
|
||||
[%- endfor %]
|
||||
default: break;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
bool endpoint_handler(int idx, cbufptr_t* input_buffer, bufptr_t* output_buffer) {
|
||||
//Introspectable property = get_property(idx);
|
||||
//if property.is_valid()
|
||||
|
||||
switch (idx) {
|
||||
[%- for endpoint in endpoints %]
|
||||
[%- if (endpoint.function.name == 'exchange' or endpoint.function.name == 'read') and endpoint.in_bindings | list == ['obj'] %]
|
||||
case [[endpoint.id]]: { return [[endpoint.function.fullname | to_snake_case]]([% for k, arg in endpoint.function.in.items() %][% if k in endpoint.in_bindings %]static_cast<[[arg.type.c_name]]>([[endpoint.in_bindings[k]]])[% else %]std::nullopt[% endif %], [% endfor %][% for k, arg in endpoint.function.out.items() %][% if k in endpoint.out_bindings %]static_cast<[[arg.type.c_name]]*>([[endpoint.out_bindings[k]]])[% else %]nullptr[% endif %], [% endfor %]input_buffer, output_buffer); } break;
|
||||
[%- else %]
|
||||
case [[endpoint.id]]: { return [[endpoint.function.fullname | to_snake_case]]([% for k, arg in endpoint.function.in.items() %][% if k in endpoint.in_bindings %]static_cast<[[arg.type.c_name]]>([[endpoint.in_bindings[k]]])[% else %]std::nullopt[% endif %], [% endfor %][% for k, arg in endpoint.function.out.items() %][% if k in endpoint.out_bindings %]static_cast<[[arg.type.c_name]]*>([[endpoint.out_bindings[k]]])[% else %]nullptr[% endif %], [% endfor %]input_buffer, output_buffer); } break;
|
||||
[%- endif %]
|
||||
[%- endfor %]
|
||||
default: return false;
|
||||
}
|
||||
}
|
||||
|
||||
bool is_endpoint_ref_valid(endpoint_ref_t endpoint_ref) {
|
||||
if (endpoint_ref.json_crc != json_crc_) {
|
||||
return false;
|
||||
}
|
||||
|
||||
switch (endpoint_ref.endpoint_id) {
|
||||
[%- for endpoint in endpoints %]
|
||||
case [[endpoint.id]]: return true;
|
||||
[%- endfor %]
|
||||
default: return false;
|
||||
}
|
||||
}
|
||||
|
||||
bool set_endpoint_from_float(endpoint_ref_t endpoint_ref, float value) {
|
||||
if (endpoint_ref.json_crc != json_crc_) {
|
||||
return false;
|
||||
}
|
||||
|
||||
Introspectable property{};
|
||||
get_property(property, endpoint_ref.endpoint_id);
|
||||
const FloatSettableTypeInfo* type_info = dynamic_cast<const FloatSettableTypeInfo*>(property.get_type_info());
|
||||
return type_info && type_info->set_float(property, value);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
#pragma GCC pop_options
|
||||
|
||||
#endif // __FIBRE_INTERFACES_HPP
|
||||
@@ -0,0 +1,40 @@
|
||||
/*[# This is the original template, thus the warning below does not apply to this file #]
|
||||
* ============================ WARNING ============================
|
||||
* ==== This is an autogenerated file. ====
|
||||
* ==== Any changes to this file will be lost when recompiling. ====
|
||||
* =================================================================
|
||||
*
|
||||
* This file contains serializing/deserializing stubs for the functions defined
|
||||
* in your interface file.
|
||||
*
|
||||
*/
|
||||
|
||||
#include <fibre/bufptr.hpp>
|
||||
|
||||
[% for intf in interfaces.values() %]
|
||||
[% for func in intf.functions.values() %]
|
||||
static inline bool [[func.fullname | to_snake_case]]([% for arg in func.in.values() %]std::optional<[[arg.type.c_name]]> in_[[arg.name]], [% endfor %][% for arg in func.out.values() %][[arg.type.c_name]]* out_[[arg.name]], [% endfor %]fibre::cbufptr_t* input_buffer, fibre::bufptr_t* output_buffer) {
|
||||
[%- if func.in %]
|
||||
bool success = [% for arg in func.in.values() %](in_[[arg.name]].has_value() || (in_[[arg.name]] = fibre::Codec<[[arg.type.c_name]]>::decode(input_buffer)).has_value()[% if arg.optional %] || true[% endif %])[% if not loop.last %]
|
||||
&& [% endif %][% endfor %];
|
||||
[%- else %]
|
||||
bool success = true;
|
||||
[%- endif %]
|
||||
if (!success) {
|
||||
return false;
|
||||
}
|
||||
[%- if func.implementation %]
|
||||
[% if func.out %]std::tuple<[% for arg in func.out.values() %][[arg.type.c_name]][[', ' if not loop.last]][% endfor %]> ret = [% endif %][[func.implementation]]([% for arg in func.in.values() %]in_[[arg.name]][% if not arg.optional %].value()[% endif %][[', ' if not loop.last]][% endfor %]);
|
||||
[%- else %]
|
||||
[% if func.out %]std::tuple<[% for arg in func.out.values() %][[arg.type.c_name]][[', ' if not loop.last]][% endfor %]> ret = [% endif %]in_[[(func.in.values() | first).name]].value()->[[func.name]]([% for arg in func.in.values() | skip_first %]in_[[arg.name]][% if not arg.optional %].value()[% endif %][[', ' if not loop.last]][% endfor %]);
|
||||
[%- endif %]
|
||||
[%- if func.out %]
|
||||
return [% for arg in func.out.values() %]((out_[[arg.name]] && ((*out_[[arg.name]] = std::get<[[loop.index0]]>(ret)), true)) || fibre::Codec<[[arg.type.c_name]]>::encode(std::get<[[loop.index0]]>(ret), output_buffer))[% if not loop.last %]
|
||||
&& [% endif %][% endfor %];
|
||||
[%- else %]
|
||||
return true;
|
||||
[%- endif %]
|
||||
}
|
||||
[% endfor %]
|
||||
[% endfor %]
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
#ifndef __FIBRE_BUFPTR_HPP
|
||||
#define __FIBRE_BUFPTR_HPP
|
||||
|
||||
namespace fibre {
|
||||
|
||||
static inline bool soft_assert(bool expr) { return expr; } // TODO: implement
|
||||
|
||||
/**
|
||||
* @brief Holds a reference to a buffer and a length.
|
||||
* Since this class implements begin() and end(), you can use it with many
|
||||
* standard algorithms that operate on iterable objects.
|
||||
*/
|
||||
template<typename T>
|
||||
struct generic_bufptr_t {
|
||||
using iterator = T*;
|
||||
using const_iterator = const T*;
|
||||
|
||||
generic_bufptr_t(T* begin, size_t length) : begin_(begin), end_(begin + length) {}
|
||||
|
||||
generic_bufptr_t(T* begin, T* end) : begin_(begin), end_(end) {}
|
||||
|
||||
generic_bufptr_t() : begin_(nullptr), end_(nullptr) {}
|
||||
|
||||
template<size_t I>
|
||||
generic_bufptr_t(T (&begin)[I]) : generic_bufptr_t(begin, I) {}
|
||||
|
||||
generic_bufptr_t(const std::vector<std::remove_const_t<T>>& vector)
|
||||
: generic_bufptr_t(vector.data(), vector.size()) {}
|
||||
|
||||
generic_bufptr_t(const generic_bufptr_t<std::remove_const_t<T>>& other)
|
||||
: generic_bufptr_t(other.begin_, other.end_) {}
|
||||
|
||||
generic_bufptr_t& operator+=(size_t num) {
|
||||
if (!soft_assert(num <= size())) {
|
||||
num = size();
|
||||
}
|
||||
begin_ += num;
|
||||
return *this;
|
||||
}
|
||||
|
||||
generic_bufptr_t operator++(int) {
|
||||
generic_bufptr_t result = *this;
|
||||
*this += 1;
|
||||
return result;
|
||||
}
|
||||
|
||||
T& operator*() {
|
||||
return *begin_;
|
||||
}
|
||||
|
||||
generic_bufptr_t take(size_t num) const {
|
||||
if (!soft_assert(num <= size())) {
|
||||
num = size();
|
||||
}
|
||||
generic_bufptr_t result = {begin_, num};
|
||||
return result;
|
||||
}
|
||||
|
||||
generic_bufptr_t skip(size_t num, size_t* processed_bytes = nullptr) const {
|
||||
if (!soft_assert(num <= size())) {
|
||||
num = size();
|
||||
}
|
||||
if (processed_bytes)
|
||||
(*processed_bytes) += num;
|
||||
return {begin_ + num, end_};
|
||||
}
|
||||
|
||||
size_t size() const {
|
||||
return end_ - begin_;
|
||||
}
|
||||
|
||||
bool empty() const {
|
||||
return size() == 0;
|
||||
}
|
||||
|
||||
T*& begin() { return begin_; }
|
||||
T*& end() { return end_; }
|
||||
T* const & begin() const { return begin_; }
|
||||
T* const & end() const { return end_; }
|
||||
T& front() const { return *begin(); }
|
||||
T& back() const { return *(end() - 1); }
|
||||
T& operator[](size_t idx) { return *(begin() + idx); }
|
||||
|
||||
T* begin_;
|
||||
T* end_;
|
||||
};
|
||||
|
||||
using cbufptr_t = generic_bufptr_t<const unsigned char>;
|
||||
using bufptr_t = generic_bufptr_t<unsigned char>;
|
||||
|
||||
}
|
||||
|
||||
#endif // __FIBRE_BUFPTR_HPP
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,56 @@
|
||||
#ifndef __CRC_HPP
|
||||
#define __CRC_HPP
|
||||
|
||||
#include <stdint.h>
|
||||
#include <limits.h>
|
||||
|
||||
// Calculates an arbitrary CRC for one byte.
|
||||
// Adapted from https://barrgroup.com/Embedded-Systems/How-To/CRC-Calculation-C-Code
|
||||
template<typename T, unsigned POLYNOMIAL>
|
||||
static T calc_crc(T remainder, uint8_t value) {
|
||||
constexpr T BIT_WIDTH = (CHAR_BIT * sizeof(T));
|
||||
constexpr T TOPBIT = ((T)1 << (BIT_WIDTH - 1));
|
||||
|
||||
// Bring the next byte into the remainder.
|
||||
remainder ^= (value << (BIT_WIDTH - 8));
|
||||
|
||||
// Perform modulo-2 division, a bit at a time.
|
||||
for (uint8_t bit = 8; bit; --bit) {
|
||||
if (remainder & TOPBIT) {
|
||||
remainder = (remainder << 1) ^ POLYNOMIAL;
|
||||
} else {
|
||||
remainder = (remainder << 1);
|
||||
}
|
||||
}
|
||||
|
||||
return remainder;
|
||||
}
|
||||
|
||||
template<typename T, unsigned POLYNOMIAL>
|
||||
static T calc_crc(T remainder, const uint8_t* buffer, size_t length) {
|
||||
while (length--)
|
||||
remainder = calc_crc<T, POLYNOMIAL>(remainder, *(buffer++));
|
||||
return remainder;
|
||||
}
|
||||
|
||||
template<unsigned POLYNOMIAL>
|
||||
static uint8_t calc_crc8(uint8_t remainder, uint8_t value) {
|
||||
return calc_crc<uint8_t, POLYNOMIAL>(remainder, value);
|
||||
}
|
||||
|
||||
template<unsigned POLYNOMIAL>
|
||||
static uint16_t calc_crc16(uint16_t remainder, uint8_t value) {
|
||||
return calc_crc<uint16_t, POLYNOMIAL>(remainder, value);
|
||||
}
|
||||
|
||||
template<unsigned POLYNOMIAL>
|
||||
static uint8_t calc_crc8(uint8_t remainder, const uint8_t* buffer, size_t length) {
|
||||
return calc_crc<uint8_t, POLYNOMIAL>(remainder, buffer, length);
|
||||
}
|
||||
|
||||
template<unsigned POLYNOMIAL>
|
||||
static uint16_t calc_crc16(uint16_t remainder, const uint8_t* buffer, size_t length) {
|
||||
return calc_crc<uint16_t, POLYNOMIAL>(remainder, buffer, length);
|
||||
}
|
||||
|
||||
#endif /* __CRC_HPP */
|
||||
@@ -0,0 +1,336 @@
|
||||
|
||||
#ifndef __DECODERS_HPP
|
||||
#define __DECODERS_HPP
|
||||
|
||||
#include "protocol.hpp"
|
||||
#include "crc.hpp"
|
||||
#include "cpp_utils.hpp"
|
||||
#include <utility>
|
||||
|
||||
|
||||
/* Base classes --------------------------------------------------------------*/
|
||||
|
||||
// @brief Base class for stream based decoders.
|
||||
// A stream based decoder is a decoder that processes arbitrary length data blocks.
|
||||
class StreamDecoder : public StreamSink {
|
||||
public:
|
||||
// @brief Returns 0 if no error ocurred, otherwise a non-zero error code.
|
||||
// Once process_bytes returned an error, subsequent calls to get_status must return the same error.
|
||||
// If the decoder is in an error state, the behavior of get_expected_bytes and process_bytes is undefined.
|
||||
virtual int get_status() = 0;
|
||||
|
||||
// @brief Returns the minimum number of bytes that are still needed to complete this decoder.
|
||||
// If 0, the decoder is considered complete and any subsequent call to process_bytes must process
|
||||
// exactly 0 bytes.
|
||||
// process_bytes() must always process all provided bytes unless the decoder expects no more bytes
|
||||
// afterwards
|
||||
virtual size_t get_expected_bytes() = 0;
|
||||
};
|
||||
|
||||
// @brief Base class for a decoder that is fed in a block-wise fashion.
|
||||
// This base class is provided for convenience when implementing certain types of decoders.
|
||||
// A StreamDecoder can be obtained from a BlockDecoder by using StreamDecoder_from_BlockDecoder.
|
||||
template<unsigned BLOCKSIZE>
|
||||
class BlockDecoder {
|
||||
public:
|
||||
typedef std::integral_constant<size_t, BLOCKSIZE> block_size;
|
||||
|
||||
virtual int get_status() = 0;
|
||||
virtual size_t get_expected_blocks() = 0;
|
||||
virtual int process_block(const uint8_t block[BLOCKSIZE]) = 0;
|
||||
private:
|
||||
};
|
||||
|
||||
// @brief Base class for a decoder that is fed in a byte-wise fashion
|
||||
// This base class is provided for convenience when implementing certain types of decoders.
|
||||
// A StreamDecoder can be obtained from a ByteDecoder by using StreamDecoder_from_ByteDecoder.
|
||||
class ByteDecoder {
|
||||
public:
|
||||
virtual int get_status() = 0;
|
||||
virtual size_t get_expected_bytes() = 0;
|
||||
virtual int process_byte(uint8_t byte) = 0;
|
||||
};
|
||||
|
||||
/* Converter classes ---------------------------------------------------------*/
|
||||
|
||||
// @brief Encapsulates a BlockDecoder to make it look like a StreamDecoder
|
||||
// @tparam T The encapsulated BlockDecoder type.
|
||||
// Must inherit from BlockDecoder.
|
||||
template<typename T, ENABLE_IF(TypeChecker<T>::template all_are<BlockDecoder<T::block_size::value>>())>
|
||||
class StreamDecoder_from_BlockDecoder : public StreamDecoder {
|
||||
public:
|
||||
// @brief Imitates the constructor signature of the encapsulated type.
|
||||
template<typename ... Args, ENABLE_IF(TypeChecker<Args...>::template first_is_not<StreamDecoder_from_BlockDecoder>())>
|
||||
explicit StreamDecoder_from_BlockDecoder(Args&& ... args)
|
||||
: block_decoder_(std::forward<Args>(args)...) {
|
||||
EXPECT_TYPE(T, BlockDecoder<T::block_size::value>);
|
||||
}
|
||||
|
||||
inline int get_status() final {
|
||||
return block_decoder_.get_status();
|
||||
}
|
||||
|
||||
inline size_t get_expected_bytes() final {
|
||||
size_t expected_bytes = block_decoder_.get_expected_blocks() * T::block_size::value;
|
||||
return expected_bytes - std::min(expected_bytes, buffer_pos_);
|
||||
}
|
||||
|
||||
inline int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) final {
|
||||
while (!get_status() && get_expected_bytes() && length) {
|
||||
// use the incoming bytes to fill internal buffer to get a complete block
|
||||
size_t n_copy = std::min(length, T::block_size::value - buffer_pos_);
|
||||
memcpy(buffer_ + buffer_pos_, buffer, n_copy);
|
||||
buffer += n_copy;
|
||||
length -= n_copy;
|
||||
if (processed_bytes) (*processed_bytes) += n_copy;
|
||||
buffer_pos_ += n_copy;
|
||||
|
||||
// if we have a full block, process it
|
||||
if (buffer_pos_ == T::block_size::value) {
|
||||
block_decoder_.process_block(buffer_);
|
||||
buffer_pos_ = 0;
|
||||
}
|
||||
}
|
||||
return get_status();
|
||||
}
|
||||
|
||||
size_t get_free_space() { return SIZE_MAX; } // TODO: deprecate
|
||||
private:
|
||||
T block_decoder_;
|
||||
size_t buffer_pos_ = 0;
|
||||
uint8_t buffer_[T::block_size::value];
|
||||
};
|
||||
|
||||
// @brief Encapsulates a ByteDecoder to make it look like a BlockDecoder
|
||||
// @tparam T The encapsulated ByteDecoder type.
|
||||
// Must inherit from ByteDecoder.
|
||||
template<typename T, ENABLE_IF(TypeChecker<T>::template all_are<ByteDecoder>())>
|
||||
class BlockDecoder_from_ByteDecoder : public BlockDecoder<1> {
|
||||
public:
|
||||
// @brief Imitates the constructor signature of the encapsulated type.
|
||||
template<typename ... Args, ENABLE_IF(TypeChecker<Args...>::template first_is_not<BlockDecoder_from_ByteDecoder>())>
|
||||
BlockDecoder_from_ByteDecoder(Args&& ... args)
|
||||
: byte_decoder_(std::forward<Args>(args)...) {
|
||||
EXPECT_TYPE(T, ByteDecoder);
|
||||
}
|
||||
|
||||
inline int get_status() final {
|
||||
return byte_decoder_.get_status();
|
||||
}
|
||||
inline size_t get_expected_blocks() final {
|
||||
return byte_decoder_.get_expected_bytes();
|
||||
}
|
||||
inline int process_block(const uint8_t block[1]) final {
|
||||
int status = byte_decoder_.process_byte(*block);
|
||||
return status;
|
||||
}
|
||||
private:
|
||||
T byte_decoder_;
|
||||
};
|
||||
|
||||
// @brief Encapsulates a ByteDecoder to make it look like a StreamDecoder
|
||||
// @tparam T The encapsulated ByteDecoder type.
|
||||
// Must inherit from ByteDecoder.
|
||||
template<typename T, ENABLE_IF(TypeChecker<T>::template all_are<ByteDecoder>())>
|
||||
class StreamDecoder_from_ByteDecoder : public StreamDecoder {
|
||||
public:
|
||||
// @brief Imitates the constructor signature of the encapsulated type.
|
||||
template<typename ... Args, ENABLE_IF(TypeChecker<Args...>::template first_is_not<StreamDecoder_from_ByteDecoder>())>
|
||||
StreamDecoder_from_ByteDecoder(Args&& ... args)
|
||||
: byte_decoder_(std::forward<Args>(args)...) {
|
||||
EXPECT_TYPE(T, ByteDecoder);
|
||||
}
|
||||
|
||||
inline int get_status() final {
|
||||
return byte_decoder_.get_status();
|
||||
}
|
||||
inline size_t get_expected_bytes() final {
|
||||
return byte_decoder_.get_expected_bytes();
|
||||
}
|
||||
inline size_t get_free_space() { return SIZE_MAX; }
|
||||
inline int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) final {
|
||||
while (!byte_decoder_.get_status() && byte_decoder_.get_expected_bytes() && length) {
|
||||
length--;
|
||||
if (processed_bytes) (*processed_bytes)++;
|
||||
byte_decoder_.process_byte(*(buffer++));
|
||||
}
|
||||
return byte_decoder_.get_status();
|
||||
}
|
||||
private:
|
||||
T byte_decoder_;
|
||||
};
|
||||
|
||||
/* Decoder implementations ---------------------------------------------------*/
|
||||
|
||||
template<typename T>
|
||||
class VarintByteDecoder : public ByteDecoder {
|
||||
public:
|
||||
static constexpr T BIT_WIDTH = (CHAR_BIT * sizeof(T));
|
||||
|
||||
VarintByteDecoder(T& state_variable) :
|
||||
state_variable_(state_variable)
|
||||
{
|
||||
}
|
||||
|
||||
size_t get_expected_bytes() final {
|
||||
return done_ ? 0 : 1;
|
||||
}
|
||||
|
||||
int get_status() final {
|
||||
return status_;
|
||||
}
|
||||
|
||||
int process_byte(uint8_t input_byte) final {
|
||||
if (bit_pos_ == 0) {
|
||||
LOG_FIBRE("start decoding varint, with 0x%02x => %zx\n", input_byte, (uintptr_t)&state_variable_);
|
||||
state_variable_ = 0;
|
||||
}
|
||||
LOG_FIBRE("varint: decode %02x << %zu at %zx\n", input_byte, bit_pos_, &bit_pos_);
|
||||
// we assume bit_pos_ < BIT_WIDTH
|
||||
state_variable_ |= (static_cast<T>(input_byte & 0x7f) << bit_pos_);
|
||||
if (((state_variable_ >> bit_pos_) & 0x7f) != static_cast<T>(input_byte & 0x7f)) {
|
||||
LOG_FIBRE("varint overflow: tried to add %02x << %zu\n", input_byte, bit_pos_);
|
||||
return (status_ = -1); // overflow
|
||||
}
|
||||
bit_pos_ += 7;
|
||||
done_ = !(input_byte & 0x80);
|
||||
return (status_ = (done_ || bit_pos_ < BIT_WIDTH) ? 0 : -1);
|
||||
}
|
||||
|
||||
private:
|
||||
T& state_variable_;
|
||||
// At all times where status_ != 0 the following statement holds:
|
||||
// (done_ || bit_pos_ < BIT_WIDTH)
|
||||
//size_t bit_pos_ = 0; // bit position
|
||||
size_t bit_pos_ = 0; // bit position
|
||||
int status_ = 0;
|
||||
bool done_ = false;
|
||||
int data[1024] = {0};
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
using VarintStreamDecoder = StreamDecoder_from_ByteDecoder<VarintByteDecoder<T>>;
|
||||
|
||||
// This double nested type should work identically but makes it way harder for the compiler to optimize
|
||||
//template<typename T>
|
||||
//using VarintBlockDecoder = BlockDecoder_from_ByteDecoder<VarintByteDecoder<T>>;
|
||||
//template<typename T>
|
||||
//using VarintStreamDecoder = StreamDecoder_from_BlockDecoder<VarintBlockDecoder<T>>;
|
||||
|
||||
template<typename T>
|
||||
inline VarintStreamDecoder<T> make_varint_decoder(T& variable) {
|
||||
return VarintStreamDecoder<T>(variable);
|
||||
}
|
||||
|
||||
inline VarintStreamDecoder<GET_TYPE_OF(&ReceiverState::endpoint_id)> make_endpoint_id_decoder(ReceiverState& state) {
|
||||
return make_varint_decoder(state.endpoint_id);
|
||||
}
|
||||
inline VarintStreamDecoder<GET_TYPE_OF(&ReceiverState::length)> make_length_decoder(ReceiverState& state) {
|
||||
return make_varint_decoder(state.length);
|
||||
}
|
||||
|
||||
|
||||
|
||||
template<uint8_t INIT, uint8_t POLYNOMIAL, typename TDecoder,
|
||||
ENABLE_IF(TypeChecker<TDecoder>::template all_are<StreamDecoder>())>
|
||||
class CRC8BlockDecoder : public BlockDecoder<CRC8_BLOCKSIZE> {
|
||||
public:
|
||||
CRC8BlockDecoder(TDecoder&& inner_decoder) :
|
||||
inner_decoder_(std::forward<TDecoder>(inner_decoder)) {
|
||||
}
|
||||
|
||||
int get_status() final {
|
||||
return status_;
|
||||
}
|
||||
|
||||
size_t get_expected_blocks() final {
|
||||
return (inner_decoder_.get_expected_bytes() + CRC8_BLOCKSIZE - 2) / (CRC8_BLOCKSIZE - 1);
|
||||
}
|
||||
|
||||
int process_block(const uint8_t input_block[4]) final {
|
||||
current_crc_ = calc_crc8<POLYNOMIAL>(current_crc_, input_block, CRC8_BLOCKSIZE - 1);
|
||||
if (current_crc_ != input_block[CRC8_BLOCKSIZE - 1])
|
||||
return status_ = -1;
|
||||
return status_ = inner_decoder_.process_bytes(input_block, CRC8_BLOCKSIZE - 1, nullptr);
|
||||
}
|
||||
private:
|
||||
TDecoder inner_decoder_;
|
||||
int status_ = 0;
|
||||
uint8_t current_crc_ = INIT;
|
||||
};
|
||||
|
||||
template<unsigned INIT, unsigned POLYNOMIAL, typename TDecoder>
|
||||
using CRC8StreamDecoder = StreamDecoder_from_BlockDecoder<CRC8BlockDecoder<INIT, POLYNOMIAL, TDecoder>>;
|
||||
|
||||
template<unsigned INIT, unsigned POLYNOMIAL, typename TDecoder>
|
||||
inline CRC8StreamDecoder<INIT, POLYNOMIAL, TDecoder> make_crc8_decoder(TDecoder&& decoder) {
|
||||
return CRC8StreamDecoder<INIT, POLYNOMIAL, TDecoder>(std::forward<TDecoder>(decoder));
|
||||
}
|
||||
|
||||
// TODO: ENABLE_IF(TypeChecker<TDecoders...>::template all_are<StreamDecoder>())
|
||||
template<typename ... TDecoders>
|
||||
class DecoderChain;
|
||||
|
||||
template<>
|
||||
class DecoderChain<> : public StreamDecoder {
|
||||
public:
|
||||
size_t get_expected_bytes() { return 0; }
|
||||
int get_status() { return 0; }
|
||||
int process_bytes(const uint8_t *input, size_t length, size_t* processed_bytes) { return 0; }
|
||||
size_t get_free_space() { return SIZE_MAX; } // TODO: deprecate
|
||||
};
|
||||
|
||||
template<typename TDecoder, typename ... TDecoders>
|
||||
class DecoderChain<TDecoder, TDecoders...> : public StreamDecoder {
|
||||
public:
|
||||
DecoderChain(TDecoder&& this_decoder, TDecoders&& ... subsequent_decoders) :
|
||||
this_decoder_(std::forward<TDecoder>(this_decoder)),
|
||||
subsequent_decoders_(std::forward<TDecoders>(subsequent_decoders)...)
|
||||
{
|
||||
EXPECT_TYPE(TDecoder, StreamDecoder);
|
||||
}
|
||||
|
||||
int get_status() final {
|
||||
// If this decoder or any of the subsequent decoders failed, return error code.
|
||||
int this_status = this_decoder_.get_status();
|
||||
int subsequent_status = subsequent_decoders_.get_status();
|
||||
if (this_status)
|
||||
return this_status;
|
||||
else if (subsequent_status)
|
||||
return subsequent_status;
|
||||
else
|
||||
return 0;
|
||||
}
|
||||
|
||||
size_t get_expected_bytes() final {
|
||||
return this_decoder_.get_expected_bytes() + subsequent_decoders_.get_expected_bytes();
|
||||
}
|
||||
|
||||
int process_bytes(const uint8_t *input, size_t length, size_t* processed_bytes) final {
|
||||
if (this_decoder_.get_expected_bytes()) {
|
||||
LOG_FIBRE("decoder chain: process %zu bytes in segment %s\n", length, typeid(TDecoder).name());
|
||||
size_t chunk = 0;
|
||||
int status = this_decoder_.process_bytes(input, length, &chunk);
|
||||
input += chunk;
|
||||
length -= chunk;
|
||||
if (processed_bytes) (*processed_bytes) += chunk;
|
||||
if (status)
|
||||
return status;
|
||||
if (!length)
|
||||
return 0;
|
||||
}
|
||||
return subsequent_decoders_.process_bytes(input, length, processed_bytes);
|
||||
}
|
||||
|
||||
size_t get_free_space() { return SIZE_MAX; } // TODO: deprecate
|
||||
private:
|
||||
TDecoder this_decoder_;
|
||||
DecoderChain<TDecoders...> subsequent_decoders_;
|
||||
};
|
||||
|
||||
template<typename ... TDecoders>
|
||||
inline DecoderChain<TDecoders...> make_decoder_chain(TDecoders&& ... decoders) {
|
||||
return DecoderChain<TDecoders...>(std::forward<TDecoders>(decoders)...);
|
||||
}
|
||||
|
||||
#endif // __DECODERS_HPP
|
||||
@@ -0,0 +1,323 @@
|
||||
|
||||
#ifndef __ENCODERS_HPP
|
||||
#define __ENCODERS_HPP
|
||||
|
||||
#include "protocol.hpp"
|
||||
#include "crc.hpp"
|
||||
#include "cpp_utils.hpp"
|
||||
#include <utility>
|
||||
|
||||
struct Request {
|
||||
endpoint_id_t endpoint_id;
|
||||
size_t length;
|
||||
};
|
||||
|
||||
/* Base classes --------------------------------------------------------------*/
|
||||
|
||||
// @brief Base class for all stream encoders
|
||||
// A stream based encoder is an encoder that generates arbitrary length data blocks.
|
||||
class StreamEncoder : public StreamSource {
|
||||
public:
|
||||
// @brief Returns 0 if no error ocurred, otherwise a non-zero error code.
|
||||
// Once get_bytes returned an error, subsequent calls to get_status must return the same error.
|
||||
// If the encoder is in an error state, the behavior of get_available_bytes and get_bytes is undefined.
|
||||
virtual int get_status() = 0;
|
||||
|
||||
// @brief Returns the minimum number of bytes that will still be generated by this encoder.
|
||||
// If 0, the encoder is considered complete and any subsequent call to get_bytes must generate
|
||||
// exactly 0 bytes.
|
||||
// get_bytes() must always generate as many bytes as requested unless the encoder generates no more bytes
|
||||
// afterwards
|
||||
virtual size_t get_available_bytes() = 0;
|
||||
};
|
||||
|
||||
// @brief Base class for an encoder that is fed in a block-wise fashion.
|
||||
// This base class is provided for convenience when implementing certain types of encoders.
|
||||
// A StreamEncoder can be obtained from a BlockEncoder by using StreamEncoder_from_BlockEncoder.
|
||||
template<unsigned BLOCKSIZE>
|
||||
class BlockEncoder {
|
||||
public:
|
||||
typedef std::integral_constant<size_t, BLOCKSIZE> block_size;
|
||||
|
||||
virtual int get_status() = 0;
|
||||
virtual size_t get_available_blocks() = 0;
|
||||
virtual int get_block(uint8_t block[BLOCKSIZE]) = 0;
|
||||
private:
|
||||
};
|
||||
|
||||
// @brief Base class for an encoder that is fed in a byte-wise fashion
|
||||
// This base class is provided for convenience when implementing certain types of encoders.
|
||||
// A StreamEncoder can be obtained from a ByteEncoder by using StreamEncoder_from_ByteEncoder.
|
||||
class ByteEncoder {
|
||||
public:
|
||||
virtual int get_status() = 0;
|
||||
virtual size_t get_available_bytes() = 0;
|
||||
virtual int get_byte(uint8_t *output_byte) = 0;
|
||||
};
|
||||
|
||||
/* Converter classes ---------------------------------------------------------*/
|
||||
|
||||
// @brief Encapsulates a BlockEncoder to make it look like a StreamEncoder
|
||||
// @tparam T The encapsulated BlockEncoder type.
|
||||
// Must inherit from to BlockEncoder.
|
||||
template<typename T, ENABLE_IF(TypeChecker<T>::template all_are<BlockEncoder<T::block_size::value>>())>
|
||||
class StreamEncoder_from_BlockEncoder : public StreamEncoder {
|
||||
public:
|
||||
// @brief Imitates the constructor signature of the encapsulated type.
|
||||
template<typename ... Args, ENABLE_IF(TypeChecker<Args...>::template first_is_not<StreamEncoder_from_BlockEncoder>())>
|
||||
explicit StreamEncoder_from_BlockEncoder(Args&& ... args)
|
||||
: block_encoder_(std::forward<Args>(args)...) {
|
||||
EXPECT_TYPE(T, BlockEncoder<T::block_size::value>);
|
||||
}
|
||||
|
||||
inline int get_status() final {
|
||||
return buffered_bytes_ ? 0 : block_encoder_.get_status();
|
||||
}
|
||||
|
||||
inline size_t get_available_bytes() final {
|
||||
size_t available_bytes = block_encoder_.get_available_blocks() * T::block_size::value;
|
||||
return available_bytes + buffered_bytes_;
|
||||
}
|
||||
|
||||
inline int get_bytes(uint8_t* buffer, size_t length, size_t* generated_bytes) final {
|
||||
while (!get_status() && get_available_bytes() && length) {
|
||||
// if the buffer is empty, retrieve a new block from the encode
|
||||
if (!buffered_bytes_) {
|
||||
block_encoder_.get_block(buffer_);
|
||||
buffered_bytes_ = T::block_size::value;
|
||||
}
|
||||
|
||||
// hand the buffered bytes to the encoder
|
||||
size_t n_copy = std::min(buffered_bytes_, length);
|
||||
memcpy(buffer, buffer_ + T::block_size::value - n_copy, n_copy);
|
||||
length -= n_copy;
|
||||
buffer += n_copy;
|
||||
if (generated_bytes) (*generated_bytes) += n_copy;
|
||||
buffered_bytes_ -= n_copy;
|
||||
}
|
||||
return get_status();
|
||||
}
|
||||
private:
|
||||
T block_encoder_;
|
||||
size_t buffered_bytes_ = 0;
|
||||
uint8_t buffer_[T::block_size::value];
|
||||
};
|
||||
|
||||
// @brief Encapsulates a ByteEncoder to make it look like a BlockEncoder
|
||||
// @tparam T The encapsulated ByteEncoder type.
|
||||
// Must inherit from ByteEncoder.
|
||||
template<typename T, ENABLE_IF(TypeChecker<T>::template all_are<ByteEncoder>())>
|
||||
class BlockEncoder_from_ByteEncoder : public BlockEncoder<1> {
|
||||
public:
|
||||
// @brief Imitates the constructor signature of the encapsulated type.
|
||||
template<typename ... Args, ENABLE_IF(TypeChecker<Args...>::template first_is_not<BlockEncoder_from_ByteEncoder>())>
|
||||
BlockEncoder_from_ByteEncoder(Args&& ... args)
|
||||
: byte_encoder_(std::forward<Args>(args)...) {
|
||||
EXPECT_TYPE(T, ByteEncoder);
|
||||
}
|
||||
|
||||
inline int get_status() final {
|
||||
return byte_encoder_.get_status();
|
||||
}
|
||||
inline size_t get_available_blocks() final {
|
||||
return byte_encoder_.get_available_bytes();
|
||||
}
|
||||
inline int get_block(uint8_t block[1]) final {
|
||||
int status = byte_encoder_.get_byte(*block);
|
||||
return status;
|
||||
}
|
||||
private:
|
||||
T byte_encoder_;
|
||||
};
|
||||
|
||||
// @brief Encapsulates a ByteEncoder to make it look like a StreamEncoder
|
||||
// @tparam T The encapsulated ByteEncoder type.
|
||||
// Must inherit from ByteEncoder.
|
||||
template<typename T, ENABLE_IF(TypeChecker<T>::template all_are<ByteEncoder>())>
|
||||
class StreamEncoder_from_ByteEncoder : public StreamEncoder {
|
||||
public:
|
||||
// @brief Imitates the constructor signature of the encapsulated type.
|
||||
template<typename ... Args, ENABLE_IF(TypeChecker<Args...>::template first_is_not<StreamEncoder_from_ByteEncoder>())>
|
||||
StreamEncoder_from_ByteEncoder(Args&& ... args)
|
||||
: byte_encoder_(std::forward<Args>(args)...) {
|
||||
EXPECT_TYPE(T, ByteEncoder);
|
||||
}
|
||||
|
||||
inline int get_status() final {
|
||||
return byte_encoder_.get_status();
|
||||
}
|
||||
inline size_t get_available_bytes() final {
|
||||
return byte_encoder_.get_available_bytes();
|
||||
}
|
||||
inline int get_bytes(uint8_t* buffer, size_t length, size_t* generated_bytes) final {
|
||||
while (!byte_encoder_.get_status() && byte_encoder_.get_available_bytes() && length) {
|
||||
length--;
|
||||
if (generated_bytes) (*generated_bytes)++;
|
||||
byte_encoder_.get_byte(buffer++);
|
||||
}
|
||||
return byte_encoder_.get_status();
|
||||
}
|
||||
private:
|
||||
T byte_encoder_;
|
||||
};
|
||||
|
||||
/* Encoder implementations ---------------------------------------------------*/
|
||||
|
||||
template<typename T>
|
||||
class VarintByteEncoder : public ByteEncoder {
|
||||
public:
|
||||
static constexpr T BIT_WIDTH = (CHAR_BIT * sizeof(T));
|
||||
|
||||
VarintByteEncoder(const T& state_variable) :
|
||||
state_variable_(state_variable)
|
||||
{}
|
||||
|
||||
size_t get_available_bytes() final {
|
||||
return done_ ? 0 : 1;
|
||||
}
|
||||
|
||||
int get_status() final {
|
||||
return 0;
|
||||
}
|
||||
|
||||
int get_byte(uint8_t *output_byte) final {
|
||||
if (bit_pos_ == 0)
|
||||
LOG_FIBRE("start encoding varint, from pos %d\n", bit_pos_);
|
||||
*output_byte = (state_variable_ >> bit_pos_) & 0x7f;
|
||||
bit_pos_ += 7;
|
||||
if (bit_pos_ < BIT_WIDTH && (state_variable_ >> bit_pos_)) {
|
||||
LOG_FIBRE("remainder: %x\n", state_variable_ >> bit_pos_);
|
||||
*output_byte |= 0x80;
|
||||
}else
|
||||
done_ = true;
|
||||
return 0;
|
||||
}
|
||||
|
||||
private:
|
||||
const T& state_variable_;
|
||||
size_t bit_pos_ = 0; // bit position
|
||||
int status_ = 0;
|
||||
bool done_ = false;
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
using VarintStreamEncoder = StreamEncoder_from_ByteEncoder<VarintByteEncoder<T>>;
|
||||
|
||||
template<typename T>
|
||||
VarintStreamEncoder<T> make_varint_encoder(const T& variable) {
|
||||
return VarintStreamEncoder<T>(variable);
|
||||
}
|
||||
|
||||
VarintStreamEncoder<GET_TYPE_OF(&Request::endpoint_id)> make_endpoint_id_encoder(const Request& request) {
|
||||
return make_varint_encoder(request.endpoint_id);
|
||||
}
|
||||
VarintStreamEncoder<GET_TYPE_OF(&Request::length)> make_length_encoder(const Request& request) {
|
||||
return make_varint_encoder(request.length);
|
||||
}
|
||||
|
||||
template<uint8_t INIT, uint8_t POLYNOMIAL, typename TEncoder,
|
||||
ENABLE_IF(TypeChecker<TEncoder>::template all_are<StreamEncoder>())>
|
||||
class CRC8BlockEncoder : public BlockEncoder<CRC8_BLOCKSIZE> {
|
||||
public:
|
||||
CRC8BlockEncoder(TEncoder&& inner_encoder)
|
||||
: inner_encoder_(std::forward<TEncoder>(inner_encoder)) {}
|
||||
|
||||
int get_status() final {
|
||||
return status_;
|
||||
}
|
||||
|
||||
size_t get_available_blocks() final {
|
||||
return (inner_encoder_.get_available_bytes() + CRC8_BLOCKSIZE - 2) / (CRC8_BLOCKSIZE - 1);
|
||||
}
|
||||
|
||||
int get_block(uint8_t block[4]) final {
|
||||
size_t generated_bytes = 0;
|
||||
status_ = inner_encoder_.get_bytes(block, CRC8_BLOCKSIZE - 1, &generated_bytes);
|
||||
if (status_)
|
||||
return status_;
|
||||
|
||||
// zero out unused end of the block
|
||||
while (generated_bytes < CRC8_BLOCKSIZE)
|
||||
block[generated_bytes++] = 0;
|
||||
|
||||
block[CRC8_BLOCKSIZE - 1] = current_crc_ = calc_crc8<POLYNOMIAL>(current_crc_, block, CRC8_BLOCKSIZE - 1);
|
||||
return 0;
|
||||
}
|
||||
private:
|
||||
TEncoder inner_encoder_;
|
||||
int status_ = 0;
|
||||
uint8_t current_crc_ = INIT;
|
||||
};
|
||||
|
||||
template<unsigned INIT, unsigned POLYNOMIAL, typename TEncoder>
|
||||
using CRC8StreamEncoder = StreamEncoder_from_BlockEncoder<CRC8BlockEncoder<INIT, POLYNOMIAL, TEncoder>>;
|
||||
|
||||
template<unsigned INIT, unsigned POLYNOMIAL, typename TEncoder>
|
||||
CRC8StreamEncoder<INIT, POLYNOMIAL, TEncoder> make_crc8_encoder(TEncoder&& encoder) {
|
||||
return CRC8StreamEncoder<INIT, POLYNOMIAL, TEncoder>(std::forward<TEncoder>(encoder));
|
||||
}
|
||||
|
||||
template<typename ... TEncoders>
|
||||
class EncoderChain;
|
||||
|
||||
template<>
|
||||
class EncoderChain<> : public StreamEncoder {
|
||||
public:
|
||||
size_t get_available_bytes() final { return 0; }
|
||||
int get_status() final { return 0; }
|
||||
int get_bytes(uint8_t *output, size_t length, size_t* generated_bytes) final { return 0; }
|
||||
};
|
||||
|
||||
template<typename TEncoder, typename ... TEncoders>
|
||||
class EncoderChain<TEncoder, TEncoders...> : public StreamEncoder {
|
||||
public:
|
||||
EncoderChain(TEncoder&& this_encoder, TEncoders&& ... subsequent_encoders) :
|
||||
this_encoder_(std::forward<TEncoder>(this_encoder)),
|
||||
subsequent_encoders_(std::forward<TEncoders>(subsequent_encoders)...)
|
||||
{
|
||||
EXPECT_TYPE(TEncoder, StreamEncoder);
|
||||
}
|
||||
|
||||
size_t get_available_bytes() final {
|
||||
return this_encoder_.get_available_bytes() + subsequent_encoders_.get_available_bytes();
|
||||
}
|
||||
|
||||
int get_status() final {
|
||||
// If this encoder or any of the subsequent encoders failed, return error code.
|
||||
int this_status = this_encoder_.get_status();
|
||||
int subsequent_status = subsequent_encoders_.get_status();
|
||||
if (this_status)
|
||||
return this_status;
|
||||
else if (subsequent_status)
|
||||
return subsequent_status;
|
||||
else
|
||||
return 0;
|
||||
}
|
||||
|
||||
int get_bytes(uint8_t *output, size_t length, size_t* generated_bytes) final {
|
||||
if (this_encoder_.get_available_bytes()) {
|
||||
LOG_FIBRE("encoder chain: generate %zu bytes in segment %s\n", length, typeid(TEncoder).name());
|
||||
size_t chunk = 0;
|
||||
int status = this_encoder_.get_bytes(output, length, &chunk);
|
||||
if (status)
|
||||
return status;
|
||||
output += chunk;
|
||||
length -= chunk;
|
||||
if (generated_bytes) *generated_bytes += chunk;
|
||||
if (!length)
|
||||
return 0;
|
||||
}
|
||||
return subsequent_encoders_.get_bytes(output, length, generated_bytes);
|
||||
}
|
||||
|
||||
private:
|
||||
TEncoder this_encoder_;
|
||||
EncoderChain<TEncoders...> subsequent_encoders_;
|
||||
};
|
||||
|
||||
template<typename ... TEncoders>
|
||||
EncoderChain<TEncoders...> make_encoder_chain(TEncoders&& ... encoders) {
|
||||
return EncoderChain<TEncoders...>(std::forward<TEncoders>(encoders)...);
|
||||
}
|
||||
|
||||
#endif // __ENCODERS_HPP
|
||||
@@ -0,0 +1,219 @@
|
||||
#ifndef __FIBRE_INTROSPECTION_HPP
|
||||
#define __FIBRE_INTROSPECTION_HPP
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <algorithm>
|
||||
#include <cstring>
|
||||
|
||||
#pragma GCC push_options
|
||||
#pragma GCC optimize ("s")
|
||||
|
||||
class TypeInfo;
|
||||
class Introspectable;
|
||||
using introspectable_storage_t = std::aligned_storage<16, 4>::type;
|
||||
|
||||
struct PropertyInfo {
|
||||
const char * name;
|
||||
const TypeInfo* type_info;
|
||||
};
|
||||
|
||||
/**
|
||||
* @brief Contains runtime accessible type information.
|
||||
*
|
||||
* Specifically, this information consists of a list of PropertyInfo items which
|
||||
* enable accessing attributes of an object by a runtime string.
|
||||
*
|
||||
* Typically, for each combination of C++ type and Fibre interface implemented
|
||||
* by this type, one (static constant) TypeInfo object will exist.
|
||||
*/
|
||||
class TypeInfo {
|
||||
friend class Introspectable;
|
||||
public:
|
||||
TypeInfo(const PropertyInfo* property_table, size_t property_table_length)
|
||||
: property_table_(property_table), property_table_length_(property_table_length) {}
|
||||
|
||||
virtual introspectable_storage_t get_child(introspectable_storage_t obj, size_t idx) const = 0;
|
||||
Introspectable get_child(const Introspectable& obj, const char * name, size_t length) const;
|
||||
|
||||
protected:
|
||||
|
||||
template<typename T> static T& as(Introspectable& obj);
|
||||
template<typename T> static const T& as(const Introspectable& obj);
|
||||
template<typename T> static Introspectable make_introspectable(T obj, const TypeInfo* type_info);
|
||||
|
||||
private:
|
||||
const PropertyInfo* property_table_;
|
||||
size_t property_table_length_;
|
||||
};
|
||||
|
||||
/**
|
||||
* @brief Wraps a reference to an application object by attaching runtime
|
||||
* accessible type information.
|
||||
*
|
||||
* The reference that is wrapped is typically a pointer but can also be a small
|
||||
* temporary, on-demand constructed object such as a fibre::Property<...> which
|
||||
* contains multiple pointers.
|
||||
*/
|
||||
class Introspectable {
|
||||
friend class TypeInfo;
|
||||
public:
|
||||
Introspectable() {}
|
||||
|
||||
/**
|
||||
* @brief Returns an Introspectable object for the attribute referenced by
|
||||
* the specified attribute name.
|
||||
*
|
||||
* The name can consist of multiple parts separated by dots.
|
||||
*
|
||||
* If the attribute does not exist, an invalid Introspectable is returned.
|
||||
*
|
||||
* @param path: The name or path of the attribute.
|
||||
* @param length: The maximum length of the name.
|
||||
*/
|
||||
Introspectable get_child(const char * path, size_t length) {
|
||||
Introspectable current = *this;
|
||||
|
||||
const char * begin = path;
|
||||
const char * end = std::find(begin, path + length, '\0');
|
||||
|
||||
while ((begin < end) && current.type_info_) {
|
||||
const char * end_of_token = std::find(begin, end, '.');
|
||||
current = current.get_direct_child(begin, end_of_token - begin);
|
||||
begin = std::min(end, end_of_token + 1);
|
||||
}
|
||||
|
||||
return current;
|
||||
};
|
||||
|
||||
bool is_valid() {
|
||||
return type_info_;
|
||||
}
|
||||
|
||||
const TypeInfo* get_type_info() {
|
||||
return type_info_;
|
||||
}
|
||||
|
||||
private:
|
||||
Introspectable get_direct_child(const char * name, size_t length) const {
|
||||
for (size_t i = 0; i < type_info_->property_table_length_; ++i) {
|
||||
if (!strncmp(name, type_info_->property_table_[i].name, length) && (length == strlen(type_info_->property_table_[i].name))) {
|
||||
Introspectable result;
|
||||
result.storage_ = type_info_->get_child(storage_, i);
|
||||
result.type_info_ = type_info_->property_table_[i].type_info;
|
||||
return result;
|
||||
}
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
public: // these should technically be protected but are public for optimization reasons
|
||||
// We use this storage to hold generic small objects. Usually that's a pointer
|
||||
// but sometimes it's an on-demand constructed Property<...>.
|
||||
// Caution: only put objects in here which are trivially copyable, movable
|
||||
// and destructible as any custom operation wouldn't be called.
|
||||
introspectable_storage_t storage_;
|
||||
const TypeInfo* type_info_ = nullptr;
|
||||
};
|
||||
|
||||
template<typename T> T& TypeInfo::as(Introspectable& obj) {
|
||||
static_assert(sizeof(T) <= sizeof(obj.storage_));
|
||||
return *(T*)&obj.storage_;
|
||||
}
|
||||
template<typename T> const T& TypeInfo::as(const Introspectable& obj) {
|
||||
static_assert(sizeof(T) <= sizeof(obj.storage_));
|
||||
return *(const T*)&obj.storage_;
|
||||
}
|
||||
template<typename T> Introspectable TypeInfo::make_introspectable(T obj, const TypeInfo* type_info) {
|
||||
Introspectable introspectable;
|
||||
as<T>(introspectable) = obj;
|
||||
introspectable.type_info_ = type_info;
|
||||
return introspectable;
|
||||
}
|
||||
|
||||
|
||||
// maybe_underlying_type_t<T> resolves to the underlying type of T if T is an enum type or otherwise to T itself.
|
||||
template<typename T, bool = std::is_enum<T>::value> struct maybe_underlying_type;
|
||||
template<typename T> struct maybe_underlying_type<T, true> { typedef std::underlying_type_t<T> type; };
|
||||
template<typename T> struct maybe_underlying_type<T, false> { typedef T type; };
|
||||
template<typename T> using maybe_underlying_type_t = typename maybe_underlying_type<T>::type;
|
||||
|
||||
|
||||
struct StringConvertibleTypeInfo {
|
||||
virtual bool get_string(const Introspectable& obj, char* buffer, size_t length) const { return false; }
|
||||
virtual bool set_string(const Introspectable& obj, char* buffer, size_t length) const { return false; }
|
||||
};
|
||||
|
||||
struct FloatSettableTypeInfo {
|
||||
//virtual bool get_float(const Introspectable& obj, float* val) const { return false; }
|
||||
virtual bool set_float(const Introspectable& obj, float val) const { return false; }
|
||||
};
|
||||
|
||||
/* Built-in type infos ********************************************************/
|
||||
|
||||
template<typename T>
|
||||
struct FibrePropertyTypeInfo;
|
||||
|
||||
// readonly property
|
||||
template<typename T>
|
||||
struct FibrePropertyTypeInfo<Property<const T>> : StringConvertibleTypeInfo, TypeInfo {
|
||||
using TypeInfo::TypeInfo;
|
||||
static const PropertyInfo property_table[];
|
||||
static const FibrePropertyTypeInfo<Property<const T>> singleton;
|
||||
|
||||
introspectable_storage_t get_child(introspectable_storage_t obj, size_t idx) const override {
|
||||
return {};
|
||||
}
|
||||
|
||||
bool get_string(const Introspectable& obj, char* buffer, size_t length) const override {
|
||||
return to_string(static_cast<maybe_underlying_type_t<T>>(as<const Property<const T>>(obj).read()), buffer, length, 0);
|
||||
}
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
const PropertyInfo FibrePropertyTypeInfo<Property<const T>>::property_table[] = {};
|
||||
template<typename T>
|
||||
const FibrePropertyTypeInfo<Property<const T>> FibrePropertyTypeInfo<Property<const T>>::singleton{FibrePropertyTypeInfo<Property<const T>>::property_table, sizeof(FibrePropertyTypeInfo<Property<const T>>::property_table) / sizeof(FibrePropertyTypeInfo<Property<const T>>::property_table[0])};
|
||||
|
||||
// readwrite property
|
||||
template<typename T>
|
||||
struct FibrePropertyTypeInfo<Property<T>> : FloatSettableTypeInfo, StringConvertibleTypeInfo, TypeInfo {
|
||||
using TypeInfo::TypeInfo;
|
||||
static const PropertyInfo property_table[];
|
||||
static const FibrePropertyTypeInfo<Property<T>> singleton;
|
||||
static const Introspectable make_introspectable(Property<T> obj) { return TypeInfo::make_introspectable(obj, &singleton); }
|
||||
|
||||
introspectable_storage_t get_child(introspectable_storage_t obj, size_t idx) const override {
|
||||
return {};
|
||||
}
|
||||
|
||||
bool get_string(const Introspectable& obj, char* buffer, size_t length) const override {
|
||||
return to_string(static_cast<maybe_underlying_type_t<T>>(as<const Property<T>>(obj).read()), buffer, length, 0);
|
||||
}
|
||||
|
||||
bool set_string(const Introspectable& obj, char* buffer, size_t length) const override {
|
||||
maybe_underlying_type_t<T> value;
|
||||
if (!from_string(buffer, length, &value, 0)) {
|
||||
return false;
|
||||
}
|
||||
as<const Property<T>>(obj).exchange(static_cast<T>(value));
|
||||
return true;
|
||||
}
|
||||
|
||||
bool set_float(const Introspectable& obj, float val) const override {
|
||||
maybe_underlying_type_t<T> value;
|
||||
if (!conversion::set_from_float(val, &value)) {
|
||||
return false;
|
||||
}
|
||||
as<const Property<T>>(obj).exchange(static_cast<T>(value));
|
||||
return true;
|
||||
}
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
const PropertyInfo FibrePropertyTypeInfo<Property<T>>::property_table[] = {};
|
||||
template<typename T>
|
||||
const FibrePropertyTypeInfo<Property<T>> FibrePropertyTypeInfo<Property<T>>::singleton{FibrePropertyTypeInfo<Property<T>>::property_table, sizeof(FibrePropertyTypeInfo<Property<T>>::property_table) / sizeof(FibrePropertyTypeInfo<Property<T>>::property_table[0])};
|
||||
|
||||
#pragma GCC pop_options
|
||||
|
||||
#endif // __FIBRE_INTROSPECTION_HPP
|
||||
@@ -0,0 +1,4 @@
|
||||
|
||||
#include "protocol.hpp"
|
||||
|
||||
int serve_on_tcp(unsigned int port);
|
||||
@@ -0,0 +1,4 @@
|
||||
|
||||
#include "protocol.hpp"
|
||||
|
||||
int serve_on_udp(unsigned int port);
|
||||
@@ -0,0 +1,621 @@
|
||||
/*
|
||||
see protocol.md for the protocol specification
|
||||
*/
|
||||
|
||||
#ifndef __PROTOCOL_HPP
|
||||
#define __PROTOCOL_HPP
|
||||
|
||||
// TODO: resolve assert
|
||||
#define assert(expr)
|
||||
|
||||
#include <functional>
|
||||
#include <limits>
|
||||
#include <cmath>
|
||||
//#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
#include <cstring>
|
||||
#include "crc.hpp"
|
||||
#include "cpp_utils.hpp"
|
||||
#include "bufptr.hpp"
|
||||
#include "simple_serdes.hpp"
|
||||
|
||||
// Note that this option cannot be used to debug UART because it prints on UART
|
||||
//#define DEBUG_FIBRE
|
||||
#ifdef DEBUG_FIBRE
|
||||
#define LOG_FIBRE(...) do { printf(__VA_ARGS__); } while (0)
|
||||
#else
|
||||
#define LOG_FIBRE(...) ((void) 0)
|
||||
#endif
|
||||
|
||||
|
||||
// Default CRC-8 Polynomial: x^8 + x^5 + x^4 + x^2 + x + 1
|
||||
// Can protect a 4 byte payload against toggling of up to 5 bits
|
||||
// source: https://users.ece.cmu.edu/~koopman/crc/index.html
|
||||
constexpr uint8_t CANONICAL_CRC8_POLYNOMIAL = 0x37;
|
||||
constexpr uint8_t CANONICAL_CRC8_INIT = 0x42;
|
||||
|
||||
constexpr size_t CRC8_BLOCKSIZE = 4;
|
||||
|
||||
// Default CRC-16 Polynomial: 0x9eb2 x^16 + x^13 + x^12 + x^11 + x^10 + x^8 + x^6 + x^5 + x^2 + 1
|
||||
// Can protect a 135 byte payload against toggling of up to 5 bits
|
||||
// source: https://users.ece.cmu.edu/~koopman/crc/index.html
|
||||
// Also known as CRC-16-DNP
|
||||
constexpr uint16_t CANONICAL_CRC16_POLYNOMIAL = 0x3d65;
|
||||
constexpr uint16_t CANONICAL_CRC16_INIT = 0x1337;
|
||||
|
||||
constexpr uint8_t CANONICAL_PREFIX = 0xAA;
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
/* move to fibre_config.h ******************************/
|
||||
|
||||
typedef size_t endpoint_id_t;
|
||||
|
||||
struct ReceiverState {
|
||||
endpoint_id_t endpoint_id;
|
||||
size_t length;
|
||||
uint16_t seqno_thread;
|
||||
uint16_t seqno;
|
||||
bool expect_ack;
|
||||
bool expect_response;
|
||||
bool enforce_ordering;
|
||||
};
|
||||
|
||||
/*******************************************************/
|
||||
|
||||
|
||||
constexpr uint16_t PROTOCOL_VERSION = 1;
|
||||
|
||||
// This value must not be larger than USB_TX_DATA_SIZE defined in usbd_cdc_if.h
|
||||
constexpr uint16_t TX_BUF_SIZE = 32; // does not work with 64 for some reason
|
||||
constexpr uint16_t RX_BUF_SIZE = 128; // larger values than 128 have currently no effect because of protocol limitations
|
||||
|
||||
// Maximum time we allocate for processing and responding to a request
|
||||
constexpr uint32_t PROTOCOL_SERVER_TIMEOUT_MS = 10;
|
||||
|
||||
|
||||
typedef struct {
|
||||
uint16_t json_crc = 0;
|
||||
uint16_t endpoint_id = 0;
|
||||
} endpoint_ref_t;
|
||||
|
||||
|
||||
namespace fibre {
|
||||
// These symbols are defined in the autogenerated endpoints.hpp
|
||||
extern const unsigned char embedded_json[];
|
||||
extern const size_t embedded_json_length;
|
||||
extern const uint16_t json_crc_;
|
||||
extern const uint32_t json_version_id_;
|
||||
bool endpoint_handler(int idx, cbufptr_t* input_buffer, bufptr_t* output_buffer);
|
||||
bool endpoint0_handler(cbufptr_t* input_buffer, bufptr_t* output_buffer);
|
||||
bool is_endpoint_ref_valid(endpoint_ref_t endpoint_ref);
|
||||
bool set_endpoint_from_float(endpoint_ref_t endpoint_ref, float value);
|
||||
}
|
||||
|
||||
|
||||
template<typename T, typename = typename std::enable_if_t<!std::is_const<T>::value>>
|
||||
inline size_t write_le(T value, uint8_t* buffer){
|
||||
//TODO: add static_assert that this is still a little endian machine
|
||||
std::memcpy(&buffer[0], &value, sizeof(value));
|
||||
return sizeof(value);
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
typename std::enable_if_t<std::is_const<T>::value, size_t>
|
||||
write_le(T value, uint8_t* buffer) {
|
||||
return write_le<std::remove_const_t<T>>(value, buffer);
|
||||
}
|
||||
|
||||
template<>
|
||||
inline size_t write_le<float>(float value, uint8_t* buffer) {
|
||||
static_assert(CHAR_BIT * sizeof(float) == 32, "32 bit floating point expected");
|
||||
static_assert(std::numeric_limits<float>::is_iec559, "IEEE 754 floating point expected");
|
||||
uint32_t value_as_uint32;
|
||||
std::memcpy(&value_as_uint32, &value, sizeof(uint32_t));
|
||||
return write_le<uint32_t>(value_as_uint32, buffer);
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
inline size_t read_le(T* value, const uint8_t* buffer){
|
||||
// TODO: add static_assert that this is still a little endian machine
|
||||
std::memcpy(value, buffer, sizeof(*value));
|
||||
return sizeof(*value);
|
||||
}
|
||||
|
||||
template<>
|
||||
inline size_t read_le<float>(float* value, const uint8_t* buffer) {
|
||||
static_assert(CHAR_BIT * sizeof(float) == 32, "32 bit floating point expected");
|
||||
static_assert(std::numeric_limits<float>::is_iec559, "IEEE 754 floating point expected");
|
||||
return read_le(reinterpret_cast<uint32_t*>(value), buffer);
|
||||
}
|
||||
|
||||
// @brief Reads a value of type T from the buffer.
|
||||
// @param buffer Pointer to the buffer to be read. The pointer is updated by the number of bytes that were read.
|
||||
// @param length The number of available bytes in buffer. This value is updated to subtract the bytes that were read.
|
||||
template<typename T>
|
||||
static inline T read_le(const uint8_t** buffer, size_t* length) {
|
||||
T result;
|
||||
size_t cnt = read_le(&result, *buffer);
|
||||
*buffer += cnt;
|
||||
*length -= cnt;
|
||||
return result;
|
||||
}
|
||||
|
||||
class PacketSink {
|
||||
public:
|
||||
// @brief Get the maximum packet length (aka maximum transmission unit)
|
||||
// A packet size shall take no action and return an error code if the
|
||||
// caller attempts to send an oversized packet.
|
||||
//virtual size_t get_mtu() = 0;
|
||||
|
||||
// @brief Processes a packet.
|
||||
// The blocking behavior shall depend on the thread-local deadline_ms variable.
|
||||
// @return: 0 on success, otherwise a non-zero error code
|
||||
// TODO: define what happens when the packet is larger than what the implementation can handle.
|
||||
virtual int process_packet(const uint8_t* buffer, size_t length) = 0;
|
||||
};
|
||||
|
||||
class StreamSink {
|
||||
public:
|
||||
// @brief Processes a chunk of bytes that is part of a continuous stream.
|
||||
// The blocking behavior shall depend on the thread-local deadline_ms variable.
|
||||
// @param processed_bytes: if not NULL, shall be incremented by the number of
|
||||
// bytes that were consumed.
|
||||
// @return: 0 on success, otherwise a non-zero error code
|
||||
virtual int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) = 0;
|
||||
|
||||
// @brief Returns the number of bytes that can still be written to the stream.
|
||||
// Shall return SIZE_MAX if the stream has unlimited lenght.
|
||||
// TODO: deprecate
|
||||
virtual size_t get_free_space() = 0;
|
||||
|
||||
/*int process_bytes(const uint8_t* buffer, size_t length) {
|
||||
size_t processed_bytes = 0;
|
||||
return process_bytes(buffer, length, &processed_bytes);
|
||||
}*/
|
||||
};
|
||||
|
||||
class StreamSource {
|
||||
public:
|
||||
// @brief Generate a chunk of bytes that are part of a continuous stream.
|
||||
// The blocking behavior shall depend on the thread-local deadline_ms variable.
|
||||
// @param generated_bytes: if not NULL, shall be incremented by the number of
|
||||
// bytes that were written to buffer.
|
||||
// @return: 0 on success, otherwise a non-zero error code
|
||||
virtual int get_bytes(uint8_t* buffer, size_t length, size_t* generated_bytes) = 0;
|
||||
|
||||
// @brief Returns the number of bytes that can still be written to the stream.
|
||||
// Shall return SIZE_MAX if the stream has unlimited lenght.
|
||||
// TODO: deprecate
|
||||
//virtual size_t get_free_space() = 0;
|
||||
};
|
||||
|
||||
class StreamToPacketSegmenter : public StreamSink {
|
||||
public:
|
||||
explicit StreamToPacketSegmenter(PacketSink& output) :
|
||||
output_(output)
|
||||
{
|
||||
};
|
||||
|
||||
int process_bytes(const uint8_t *buffer, size_t length, size_t* processed_bytes) override;
|
||||
|
||||
size_t get_free_space() { return SIZE_MAX; }
|
||||
|
||||
private:
|
||||
uint8_t header_buffer_[3] = {0};
|
||||
size_t header_index_ = 0;
|
||||
uint8_t packet_buffer_[RX_BUF_SIZE] = {0};
|
||||
size_t packet_index_ = 0;
|
||||
size_t packet_length_ = 0;
|
||||
PacketSink& output_;
|
||||
};
|
||||
|
||||
|
||||
class StreamBasedPacketSink : public PacketSink {
|
||||
public:
|
||||
explicit StreamBasedPacketSink(StreamSink& output) :
|
||||
output_(output)
|
||||
{
|
||||
};
|
||||
|
||||
//size_t get_mtu() { return SIZE_MAX; }
|
||||
int process_packet(const uint8_t *buffer, size_t length) override;
|
||||
|
||||
private:
|
||||
StreamSink& output_;
|
||||
};
|
||||
|
||||
// @brief: Represents a stream sink that's based on an underlying packet sink.
|
||||
// A single call to process_bytes may result in multiple packets being sent.
|
||||
class PacketBasedStreamSink : public StreamSink {
|
||||
public:
|
||||
explicit PacketBasedStreamSink(PacketSink& packet_sink) : _packet_sink(packet_sink) {}
|
||||
~PacketBasedStreamSink() {}
|
||||
|
||||
int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) override {
|
||||
// Loop to ensure all bytes get sent
|
||||
while (length) {
|
||||
size_t chunk = length;
|
||||
// send chunk as packet
|
||||
if (_packet_sink.process_packet(buffer, chunk))
|
||||
return -1;
|
||||
buffer += chunk;
|
||||
length -= chunk;
|
||||
if (processed_bytes)
|
||||
*processed_bytes += chunk;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
size_t get_free_space() { return SIZE_MAX; }
|
||||
|
||||
private:
|
||||
PacketSink& _packet_sink;
|
||||
};
|
||||
|
||||
// Implements the StreamSink interface by writing into a fixed size
|
||||
// memory buffer.
|
||||
class MemoryStreamSink : public StreamSink {
|
||||
public:
|
||||
MemoryStreamSink(uint8_t *buffer, size_t length) :
|
||||
buffer_(buffer),
|
||||
buffer_length_(length) {}
|
||||
|
||||
// Returns 0 on success and -1 if the buffer could not accept everything because it became full
|
||||
int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) override {
|
||||
size_t chunk = length < buffer_length_ ? length : buffer_length_;
|
||||
memcpy(buffer_, buffer, chunk);
|
||||
buffer_ += chunk;
|
||||
buffer_length_ -= chunk;
|
||||
if (processed_bytes)
|
||||
*processed_bytes += chunk;
|
||||
return chunk == length ? 0 : -1;
|
||||
}
|
||||
|
||||
size_t get_free_space() { return buffer_length_; }
|
||||
|
||||
private:
|
||||
uint8_t * buffer_;
|
||||
size_t buffer_length_;
|
||||
};
|
||||
|
||||
// Implements the StreamSink interface by discarding the first couple of bytes
|
||||
// and then forwarding the rest to another stream.
|
||||
class NullStreamSink : public StreamSink {
|
||||
public:
|
||||
NullStreamSink(size_t skip, StreamSink& follow_up_stream) :
|
||||
skip_(skip),
|
||||
follow_up_stream_(follow_up_stream) {}
|
||||
|
||||
// Returns 0 on success and -1 if the buffer could not accept everything because it became full
|
||||
int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) override {
|
||||
if (skip_ < length) {
|
||||
buffer += skip_;
|
||||
length -= skip_;
|
||||
if (processed_bytes)
|
||||
*processed_bytes += skip_;
|
||||
skip_ = 0;
|
||||
return follow_up_stream_.process_bytes(buffer, length, processed_bytes);
|
||||
} else {
|
||||
skip_ -= length;
|
||||
if (processed_bytes)
|
||||
*processed_bytes += length;
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
size_t get_free_space() override { return skip_ + follow_up_stream_.get_free_space(); }
|
||||
|
||||
private:
|
||||
size_t skip_;
|
||||
StreamSink& follow_up_stream_;
|
||||
};
|
||||
|
||||
|
||||
|
||||
// Implements the StreamSink interface by calculating the CRC16 checksum
|
||||
// on the data that is sent to it.
|
||||
class CRC16Calculator : public StreamSink {
|
||||
public:
|
||||
explicit CRC16Calculator(uint16_t crc16_init) :
|
||||
crc16_(crc16_init) {}
|
||||
|
||||
int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) override{
|
||||
crc16_ = calc_crc16<CANONICAL_CRC16_POLYNOMIAL>(crc16_, buffer, length);
|
||||
if (processed_bytes)
|
||||
*processed_bytes += length;
|
||||
return 0;
|
||||
}
|
||||
|
||||
size_t get_free_space() override { return SIZE_MAX; }
|
||||
|
||||
uint16_t get_crc16() { return crc16_; }
|
||||
private:
|
||||
uint16_t crc16_;
|
||||
};
|
||||
|
||||
|
||||
namespace fibre {
|
||||
template<typename T, typename = void>
|
||||
struct Codec {
|
||||
static std::optional<T> decode(cbufptr_t* buffer) { return std::nullopt; }
|
||||
};
|
||||
|
||||
template<> struct Codec<bool> {
|
||||
static std::optional<bool> decode(cbufptr_t* buffer) { return (buffer->begin() == buffer->end()) ? std::nullopt : std::make_optional((bool)*(buffer->begin()++)); }
|
||||
static bool encode(bool value, bufptr_t* buffer) { return SimpleSerializer<uint8_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<int8_t> {
|
||||
static std::optional<int8_t> decode(cbufptr_t* buffer) { return SimpleSerializer<int8_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(int8_t value, bufptr_t* buffer) { return SimpleSerializer<int8_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<uint8_t> {
|
||||
static std::optional<uint8_t> decode(cbufptr_t* buffer) { return SimpleSerializer<uint8_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(uint8_t value, bufptr_t* buffer) { return SimpleSerializer<uint8_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<int16_t> {
|
||||
static std::optional<int16_t> decode(cbufptr_t* buffer) { return SimpleSerializer<int16_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(int16_t value, bufptr_t* buffer) { return SimpleSerializer<int16_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<uint16_t> {
|
||||
static std::optional<uint16_t> decode(cbufptr_t* buffer) { return SimpleSerializer<uint16_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(uint16_t value, bufptr_t* buffer) { return SimpleSerializer<uint16_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<int32_t> {
|
||||
static std::optional<int32_t> decode(cbufptr_t* buffer) { return SimpleSerializer<int32_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(int32_t value, bufptr_t* buffer) { return SimpleSerializer<int32_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<uint32_t> {
|
||||
static std::optional<uint32_t> decode(cbufptr_t* buffer) { return SimpleSerializer<uint32_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(uint32_t value, bufptr_t* buffer) { return SimpleSerializer<uint32_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<int64_t> {
|
||||
static std::optional<int64_t> decode(cbufptr_t* buffer) { return SimpleSerializer<int64_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(int64_t value, bufptr_t* buffer) { return SimpleSerializer<int64_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<uint64_t> {
|
||||
static std::optional<uint64_t> decode(cbufptr_t* buffer) { return SimpleSerializer<uint64_t, false>::read(&(buffer->begin()), buffer->end()); }
|
||||
static bool encode(uint64_t value, bufptr_t* buffer) { return SimpleSerializer<uint64_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<float> {
|
||||
static std::optional<float> decode(cbufptr_t* buffer) {
|
||||
std::optional<uint32_t> int_val = Codec<uint32_t>::decode(buffer);
|
||||
return int_val.has_value() ? std::optional<float>(*reinterpret_cast<float*>(&int_val.value())) : std::nullopt;
|
||||
}
|
||||
static bool encode(float value, bufptr_t* buffer) {
|
||||
void* ptr = &value;
|
||||
return Codec<uint32_t>::encode(*reinterpret_cast<uint32_t*>(ptr), buffer);
|
||||
}
|
||||
};
|
||||
template<typename T>
|
||||
struct Codec<T, std::enable_if_t<std::is_enum<T>::value>> {
|
||||
static std::optional<T> decode(cbufptr_t* buffer) {
|
||||
std::optional<int32_t> int_val = SimpleSerializer<int32_t, false>::read(&(buffer->begin()), buffer->end());
|
||||
return int_val.has_value() ? std::make_optional(static_cast<T>(int_val.value())) : std::nullopt;
|
||||
}
|
||||
static bool encode(T value, bufptr_t* buffer) { return SimpleSerializer<int32_t, false>::write(value, &(buffer->begin()), buffer->end()); }
|
||||
};
|
||||
template<> struct Codec<endpoint_ref_t> {
|
||||
static std::optional<endpoint_ref_t> decode(cbufptr_t* buffer) {
|
||||
std::optional<uint16_t> val0 = SimpleSerializer<uint16_t, false>::read(&(buffer->begin()), buffer->end());
|
||||
std::optional<uint16_t> val1 = SimpleSerializer<uint16_t, false>::read(&(buffer->begin()), buffer->end());
|
||||
return (val0.has_value() && val1.has_value()) ? std::make_optional(endpoint_ref_t{val1.value(), val0.value()}) : std::nullopt;
|
||||
}
|
||||
static bool encode(endpoint_ref_t value, bufptr_t* buffer) {
|
||||
return SimpleSerializer<uint16_t, false>::write(value.endpoint_id, &(buffer->begin()), buffer->end())
|
||||
&& SimpleSerializer<uint16_t, false>::write(value.json_crc, &(buffer->begin()), buffer->end());
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
/* @brief Handles the communication protocol on one channel.
|
||||
*
|
||||
* When instantiated with a list of endpoints and an output packet sink,
|
||||
* objects of this class will handle packets passed into process_packet,
|
||||
* pass the relevant data to the corresponding endpoints and dispatch response
|
||||
* packets on the output.
|
||||
*/
|
||||
class BidirectionalPacketBasedChannel : public PacketSink {
|
||||
public:
|
||||
explicit BidirectionalPacketBasedChannel(PacketSink& output) :
|
||||
output_(output)
|
||||
{ }
|
||||
|
||||
//size_t get_mtu() {
|
||||
// return SIZE_MAX;
|
||||
//}
|
||||
int process_packet(const uint8_t* buffer, size_t length) override;
|
||||
private:
|
||||
PacketSink& output_;
|
||||
uint8_t tx_buf_[TX_BUF_SIZE] = {0};
|
||||
};
|
||||
|
||||
|
||||
/* ToString / FromString functions -------------------------------------------*/
|
||||
/*
|
||||
* These functions are currently not used by Fibre and only here to
|
||||
* support the ODrive ASCII protocol.
|
||||
* TODO: find a general way for client code to augment endpoints with custom
|
||||
* functions
|
||||
*/
|
||||
|
||||
template<typename T>
|
||||
struct format_traits_t;
|
||||
|
||||
// template<> struct format_traits_t<float> { using type = void;
|
||||
// static constexpr const char * fmt = "%f";
|
||||
// static constexpr const char * fmtp = "%f";
|
||||
// };
|
||||
template<> struct format_traits_t<int64_t> { using type = void;
|
||||
static constexpr const char * fmt = "%lld";
|
||||
static constexpr const char * fmtp = "%lld";
|
||||
};
|
||||
template<> struct format_traits_t<uint64_t> { using type = void;
|
||||
static constexpr const char * fmt = "%llu";
|
||||
static constexpr const char * fmtp = "%llu";
|
||||
};
|
||||
template<> struct format_traits_t<int32_t> { using type = void;
|
||||
static constexpr const char * fmt = "%ld";
|
||||
static constexpr const char * fmtp = "%ld";
|
||||
};
|
||||
template<> struct format_traits_t<uint32_t> { using type = void;
|
||||
static constexpr const char * fmt = "%lu";
|
||||
static constexpr const char * fmtp = "%lu";
|
||||
};
|
||||
// TODO: change all overloads to fundamental int type space
|
||||
template<> struct format_traits_t<unsigned int> { using type = void;
|
||||
static constexpr const char * fmt = "%ud";
|
||||
static constexpr const char * fmtp = "%ud";
|
||||
};
|
||||
template<> struct format_traits_t<int16_t> { using type = void;
|
||||
static constexpr const char * fmt = "%hd";
|
||||
static constexpr const char * fmtp = "%hd";
|
||||
};
|
||||
template<> struct format_traits_t<uint16_t> { using type = void;
|
||||
static constexpr const char * fmt = "%hu";
|
||||
static constexpr const char * fmtp = "%hu";
|
||||
};
|
||||
template<> struct format_traits_t<int8_t> { using type = void;
|
||||
static constexpr const char * fmt = "%hhd";
|
||||
static constexpr const char * fmtp = "%d";
|
||||
};
|
||||
template<> struct format_traits_t<uint8_t> { using type = void;
|
||||
static constexpr const char * fmt = "%hhu";
|
||||
static constexpr const char * fmtp = "%u";
|
||||
};
|
||||
|
||||
template<typename T, typename = typename format_traits_t<T>::type>
|
||||
static bool to_string(const T& value, char * buffer, size_t length, int) {
|
||||
snprintf(buffer, length, format_traits_t<T>::fmtp, value);
|
||||
return true;
|
||||
}
|
||||
// Special case for float because printf promotes float to double, and we get warnings
|
||||
template<typename T = float>
|
||||
static bool to_string(const float& value, char * buffer, size_t length, int) {
|
||||
snprintf(buffer, length, "%f", (double)value);
|
||||
return true;
|
||||
}
|
||||
template<typename T = bool>
|
||||
static bool to_string(const bool& value, char * buffer, size_t length, int) {
|
||||
buffer[0] = value ? '1' : '0';
|
||||
buffer[1] = 0;
|
||||
return true;
|
||||
}
|
||||
template<typename T>
|
||||
static bool to_string(const T& value, char * buffer, size_t length, ...) {
|
||||
return false;
|
||||
}
|
||||
|
||||
template<typename T, typename = typename format_traits_t<T>::type>
|
||||
static bool from_string(const char * buffer, size_t length, T* property, int) {
|
||||
// Note for T == uint8_t: Even though we supposedly use the correct format
|
||||
// string sscanf treats our pointer as pointer-to-int instead of
|
||||
// pointer-to-uint8_t. To avoid an unexpected memory access we first read
|
||||
// into a union.
|
||||
union { T t; int i; } val;
|
||||
if (sscanf(buffer, format_traits_t<T>::fmt, &val.t) == 1) {
|
||||
*property = val.t;
|
||||
return true;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// Special case for float because printf promotes float to double, and we get warnings
|
||||
template<typename T = float>
|
||||
static bool from_string(const char * buffer, size_t length, float* property, int) {
|
||||
return sscanf(buffer, "%f", property) == 1;
|
||||
}
|
||||
template<typename T = bool>
|
||||
static bool from_string(const char * buffer, size_t length, bool* property, int) {
|
||||
int val;
|
||||
if (sscanf(buffer, "%d", &val) != 1)
|
||||
return false;
|
||||
*property = val;
|
||||
return true;
|
||||
}
|
||||
template<typename T>
|
||||
static bool from_string(const char * buffer, size_t length, T* property, ...) {
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
//template<typename T, typename = typename std>
|
||||
//bool set_from_float_ex(float value, T* property) {
|
||||
// return false;
|
||||
//}
|
||||
|
||||
namespace conversion {
|
||||
//template<typename T>
|
||||
template<typename T>
|
||||
bool set_from_float_ex(float value, float* property, int) {
|
||||
return *property = value, true;
|
||||
}
|
||||
template<typename T>
|
||||
bool set_from_float_ex(float value, bool* property, int) {
|
||||
return *property = (value >= 0.0f), true;
|
||||
}
|
||||
template<typename T, typename = std::enable_if_t<std::is_integral<T>::value && !std::is_const<T>::value>>
|
||||
bool set_from_float_ex(float value, T* property, int) {
|
||||
return *property = static_cast<T>(std::round(value)), true;
|
||||
}
|
||||
template<typename T>
|
||||
bool set_from_float_ex(float value, T* property, ...) {
|
||||
return false;
|
||||
}
|
||||
template<typename T>
|
||||
bool set_from_float(float value, T* property) {
|
||||
return set_from_float_ex<T>(value, property, 0);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
template<typename T>
|
||||
struct Property {
|
||||
Property(void* ctx, T(*getter)(void*), void(*setter)(void*, T))
|
||||
: ctx_(ctx), getter_(getter), setter_(setter) {}
|
||||
Property(T* ctx)
|
||||
: ctx_(ctx), getter_([](void* ctx){ return *(T*)ctx; }), setter_([](void* ctx, T val){ *(T*)ctx = val; }) {}
|
||||
Property& operator*() { return *this; }
|
||||
Property* operator->() { return this; }
|
||||
|
||||
T read() const {
|
||||
return (*getter_)(ctx_);
|
||||
}
|
||||
|
||||
T exchange(std::optional<T> value) const {
|
||||
T old_value = (*getter_)(ctx_);
|
||||
if (value.has_value()) {
|
||||
(*setter_)(ctx_, value.value());
|
||||
}
|
||||
return old_value;
|
||||
}
|
||||
|
||||
void* ctx_;
|
||||
T(*getter_)(void*);
|
||||
void(*setter_)(void*, T);
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
struct Property<const T> {
|
||||
Property(void* ctx, T(*getter)(void*))
|
||||
: ctx_(ctx), getter_(getter) {}
|
||||
Property(const T* ctx)
|
||||
: ctx_(const_cast<T*>(ctx)), getter_([](void* ctx){ return *(const T*)ctx; }) {}
|
||||
Property& operator*() { return *this; }
|
||||
Property* operator->() { return this; }
|
||||
|
||||
T read() const {
|
||||
return (*getter_)(ctx_);
|
||||
}
|
||||
|
||||
void* ctx_;
|
||||
T(*getter_)(void*);
|
||||
};
|
||||
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,77 @@
|
||||
#ifndef __FIBRE_SIMPLE_SERDES
|
||||
#define __FIBRE_SIMPLE_SERDES
|
||||
|
||||
//#include "stream.hpp"
|
||||
|
||||
|
||||
template<typename T, bool BigEndian, typename = void>
|
||||
struct SimpleSerializer;
|
||||
template<typename T>
|
||||
using LittleEndianSerializer = SimpleSerializer<T, false>;
|
||||
template<typename T>
|
||||
using BigEndianSerializer = SimpleSerializer<T, true>;
|
||||
|
||||
|
||||
/* @brief Serializer/deserializer for arbitrary integral number types */
|
||||
// TODO: allow reading an arbitrary number of bits
|
||||
template<typename T, bool BigEndian>
|
||||
struct SimpleSerializer<T, BigEndian, typename std::enable_if_t<std::is_integral<T>::value>> {
|
||||
static constexpr size_t BIT_WIDTH = std::numeric_limits<T>::digits;
|
||||
static constexpr size_t BYTE_WIDTH = (BIT_WIDTH + 7) / 8;
|
||||
|
||||
template<typename TIterator>
|
||||
static std::optional<T> read(TIterator* begin, TIterator end = nullptr) {
|
||||
T result = 0;
|
||||
if (BigEndian) {
|
||||
for (size_t i = BYTE_WIDTH; i > 0; (i++, (*begin)++)) {
|
||||
if (end && !(*begin < end))
|
||||
return std::nullopt;
|
||||
uint8_t byte = **begin;
|
||||
result |= static_cast<T>(byte) << ((i - 1) << 3);
|
||||
}
|
||||
} else {
|
||||
for (size_t i = 0; i < BYTE_WIDTH; (i++, (*begin)++)) {
|
||||
if (end && !(*begin < end))
|
||||
return std::nullopt;
|
||||
uint8_t byte = **begin;
|
||||
result |= static_cast<T>(byte) << (i << 3);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
template<typename TIterator>
|
||||
static bool write(T value, TIterator* begin, TIterator end = nullptr) {
|
||||
if (BigEndian) {
|
||||
for (size_t i = BYTE_WIDTH; i > 0; (i--, (*begin)++)) {
|
||||
if (end && !(*begin < end))
|
||||
return false;
|
||||
uint8_t byte = static_cast<uint8_t>((value >> ((i - 1) << 3)) & 0xff);
|
||||
**begin = byte;
|
||||
}
|
||||
} else {
|
||||
for (size_t i = 0; i < BYTE_WIDTH; (i++, (*begin)++)) {
|
||||
if (end && !(*begin < end))
|
||||
return false;
|
||||
uint8_t byte = static_cast<uint8_t>((value >> (i << 3)) & 0xff);
|
||||
**begin = byte;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
};
|
||||
|
||||
template<typename T>
|
||||
inline std::optional<T> read_le(fibre::cbufptr_t* buffer) {
|
||||
static_assert(is_complete<LittleEndianSerializer<T>>(), "no LittleEndianSerializer is defined for type T");
|
||||
return LittleEndianSerializer<T>::read(&buffer->begin(), buffer->end());
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
inline bool write_le(T value, fibre::bufptr_t* buffer) {
|
||||
static_assert(is_complete<LittleEndianSerializer<T>>(), "no LittleEndianSerializer is defined for type T");
|
||||
return LittleEndianSerializer<T>::write(value, &buffer->begin(), buffer->end());
|
||||
}
|
||||
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,94 @@
|
||||
/*[# This is the original template, thus the warning below does not apply to this file #]
|
||||
* ============================ WARNING ============================
|
||||
* ==== This is an autogenerated file. ====
|
||||
* ==== Any changes to this file will be lost when recompiling. ====
|
||||
* =================================================================
|
||||
*
|
||||
* This file contains base classes that correspond to the interfaces defined in
|
||||
* your interface file. The objects you publish should inherit from these
|
||||
* interfaces.
|
||||
*
|
||||
*/
|
||||
|
||||
#pragma GCC push_options
|
||||
#pragma GCC optimize ("s")
|
||||
|
||||
[%- macro rettype(func) %]
|
||||
[%- if not func.out -%]
|
||||
void
|
||||
[%- elif func.out | length == 1 -%]
|
||||
[[(func.out.values() | first).type.c_name]]
|
||||
[%- else -%]
|
||||
[% for arg in func.out.values() %][[arg.type]][[', ' if not loop.last]][% endfor %]
|
||||
[%- endif -%]
|
||||
[%- endmacro %]
|
||||
|
||||
[%- macro render_interface(intf) %]
|
||||
class [[intf.name | to_pascal_case]]Intf {
|
||||
public:
|
||||
[%- for intf in intf.interfaces -%]
|
||||
[[render_interface(intf) | indent(4)]]
|
||||
[%- endfor %]
|
||||
[%- for enum in intf.enums %]
|
||||
enum [[enum.name | to_pascal_case]] {
|
||||
[%- for k, value in enum['values'].items() %]
|
||||
[[((enum.name + k) | to_macro_case).ljust(32)]] = [% if enum.is_flags %]0x[['%08x' | format(value.value)]][% else %][[value.value]][% endif %],
|
||||
[%- endfor %]
|
||||
};
|
||||
[%- endfor %]
|
||||
|
||||
[%- for property in intf.attributes.values() %]
|
||||
[%- if property.type.fullname.startswith("fibre.Property") %]
|
||||
[%- if not property.c_getter and not property.c_setter %]
|
||||
template<typename T> static inline auto get_[[property.name]](T* obj) { return [[property.type.c_name]]{&obj->[[property.c_name]]}; }
|
||||
template<typename T> static inline void get_[[property.name]](T* obj, void* ptr) { new (ptr) [[property.type.c_name]]{&obj->[[property.c_name]]}; }[# these are for the set_endpoint_from_float function. This is unmaintainable and should go away #]
|
||||
[%- elif not property.c_setter %]
|
||||
template<typename T> static inline auto get_[[property.name]](T* obj) { return [[property.type.c_name]]{obj, [](void* ctx){ return ([[property.type.value_type.c_name]])((T*)ctx)->[[property.c_getter]]; }}; }
|
||||
template<typename T> static inline void get_[[property.name]](T* obj, void* ptr) { new (ptr) [[property.type.c_name]]{obj, [](void* ctx){ return ([[property.type.value_type.c_name]])((T*)ctx)->[[property.c_getter]]; }}; }
|
||||
[%- else %]
|
||||
template<typename T> static inline auto get_[[property.name]](T* obj) { return [[property.type.c_name]]{obj, [](void* ctx){ return ([[property.type.value_type.c_name]])((T*)ctx)->[[property.c_getter]]; }, [](void* ctx, [[property.type.value_type.c_name]] value){ ((T*)ctx)->[[property.c_setter]](value); }}; }
|
||||
template<typename T> static inline void get_[[property.name]](T* obj, void* ptr) { new (ptr) [[property.type.c_name]]{obj, [](void* ctx){ return ([[property.type.value_type.c_name]])((T*)ctx)->[[property.c_getter]]; }, [](void* ctx, [[property.type.value_type.c_name]] value){ ((T*)ctx)->[[property.c_setter]](value); }}; }
|
||||
[%- endif %]
|
||||
[%- else %]
|
||||
template<typename T> static inline auto get_[[property.name]](T* obj) { return &obj->[[property.c_name]]; }
|
||||
[%- endif %]
|
||||
[%- endfor %]
|
||||
|
||||
[%- for func in intf.functions.values() %]
|
||||
virtual [[rettype(func)]] [[func.name | to_snake_case]]([% for in in func.in.values() %][% if loop.index0 %][[in.type.c_name]] [[in.name]][[', ' if not loop.last]][% endif %][% endfor %]) = 0;
|
||||
[%- endfor %]
|
||||
[%- for func in intf.functions.values() %]
|
||||
[%- for k, arg in func.in.items() | skip_first %]
|
||||
[[arg.type.c_name]] [[func.name | to_snake_case]]_in_[[arg.name]]_; // for internal use by Fibre
|
||||
template<typename T> static auto get_[[func.name | to_snake_case]]_in_[[arg.name]]_(T* obj) { return Property<[[arg.type.c_name]]>{&obj->[[func.name | to_snake_case]]_in_[[arg.name]]_}; }
|
||||
template<typename T> static void get_[[func.name | to_snake_case]]_in_[[arg.name]]_(T* obj, void* ptr) { new (ptr) Property<[[arg.type.c_name]]>{&obj->[[func.name | to_snake_case]]_in_[[arg.name]]_}; }
|
||||
[%- endfor %]
|
||||
[%- for k, arg in func.out.items() %]
|
||||
[[arg.type.c_name]] [[func.name | to_snake_case]]_out_[[arg.name]]_; // for internal use by Fibre
|
||||
template<typename T> static auto get_[[func.name | to_snake_case]]_out_[[arg.name]]_(T* obj) { return Property<const [[arg.type.c_name]]>{&obj->[[func.name | to_snake_case]]_out_[[arg.name]]_}; }
|
||||
template<typename T> static void get_[[func.name | to_snake_case]]_out_[[arg.name]]_(T* obj, void* ptr) { new (ptr) Property<const [[arg.type.c_name]]>{&obj->[[func.name | to_snake_case]]_out_[[arg.name]]_}; }
|
||||
[%- endfor %]
|
||||
[%- endfor %]
|
||||
};
|
||||
[%- endmacro %]
|
||||
|
||||
[% for intf in toplevel_interfaces %]
|
||||
[[render_interface(intf)]]
|
||||
[% endfor %]
|
||||
|
||||
[%- for _, enum in value_types.items() %]
|
||||
[%- if enum.is_flags %]
|
||||
// this is technically not thread-safe but practically it might be
|
||||
inline [[enum.c_name]] operator | ([[enum.c_name]] a, [[enum.c_name]] b) { return static_cast<[[enum.c_name]]>(static_cast<std::underlying_type_t<[[enum.c_name]]>>(a) | static_cast<std::underlying_type_t<[[enum.c_name]]>>(b)); }
|
||||
inline [[enum.c_name]] operator & ([[enum.c_name]] a, [[enum.c_name]] b) { return static_cast<[[enum.c_name]]>(static_cast<std::underlying_type_t<[[enum.c_name]]>>(a) & static_cast<std::underlying_type_t<[[enum.c_name]]>>(b)); }
|
||||
inline [[enum.c_name]] operator ^ ([[enum.c_name]] a, [[enum.c_name]] b) { return static_cast<[[enum.c_name]]>(static_cast<std::underlying_type_t<[[enum.c_name]]>>(a) ^ static_cast<std::underlying_type_t<[[enum.c_name]]>>(b)); }
|
||||
inline [[enum.c_name]]& operator |= ([[enum.c_name]] &a, [[enum.c_name]] b) { return reinterpret_cast<[[enum.c_name]]&>(reinterpret_cast<std::underlying_type_t<[[enum.c_name]]>&>(a) |= static_cast<std::underlying_type_t<[[enum.c_name]]>>(b)); }
|
||||
inline [[enum.c_name]]& operator &= ([[enum.c_name]] &a, [[enum.c_name]] b) { return reinterpret_cast<[[enum.c_name]]&>(reinterpret_cast<std::underlying_type_t<[[enum.c_name]]>&>(a) &= static_cast<std::underlying_type_t<[[enum.c_name]]>>(b)); }
|
||||
inline [[enum.c_name]]& operator ^= ([[enum.c_name]] &a, [[enum.c_name]] b) { return reinterpret_cast<[[enum.c_name]]&>(reinterpret_cast<std::underlying_type_t<[[enum.c_name]]>&>(a) ^= static_cast<std::underlying_type_t<[[enum.c_name]]>>(b)); }
|
||||
inline [[enum.c_name]] operator ~ ([[enum.c_name]] a) { return static_cast<[[enum.c_name]]>(~static_cast<std::underlying_type_t<[[enum.c_name]]>>(a)); }
|
||||
[%- endif %]
|
||||
[%- endfor %]
|
||||
|
||||
|
||||
|
||||
#pragma GCC pop_options
|
||||
@@ -0,0 +1,8 @@
|
||||
|
||||
tup.include('../tupfiles/build.lua')
|
||||
|
||||
fibre_package = define_package{
|
||||
sources={'protocol.cpp', 'posix_tcp.cpp', 'posix_udp.cpp'},
|
||||
libs={'pthread'},
|
||||
headers={'include'}
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
|
||||
#include <arpa/inet.h>
|
||||
#include <netinet/in.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/socket.h>
|
||||
#include <unistd.h>
|
||||
#include <thread>
|
||||
#include <future>
|
||||
#include <vector>
|
||||
|
||||
#include <fibre/protocol.hpp>
|
||||
|
||||
|
||||
#define TCP_RX_BUF_LEN 512
|
||||
|
||||
class TCPStreamSink : public StreamSink {
|
||||
public:
|
||||
TCPStreamSink(int socket_fd) :
|
||||
socket_fd_(socket_fd)
|
||||
{}
|
||||
|
||||
int process_bytes(const uint8_t* buffer, size_t length, size_t* processed_bytes) {
|
||||
int bytes_sent = send(socket_fd_, buffer, length, 0);
|
||||
if (processed_bytes)
|
||||
*processed_bytes = (bytes_sent == -1) ? 0 : bytes_sent;
|
||||
return (bytes_sent == -1) ? -1 : 0;
|
||||
}
|
||||
|
||||
size_t get_free_space() { return SIZE_MAX; }
|
||||
|
||||
private:
|
||||
int socket_fd_;
|
||||
};
|
||||
|
||||
|
||||
int serve_client(int sock_fd) {
|
||||
uint8_t buf[TCP_RX_BUF_LEN];
|
||||
|
||||
// initialize output stack for this client
|
||||
TCPStreamSink tcp_packet_output(sock_fd);
|
||||
StreamBasedPacketSink packet2stream(tcp_packet_output);
|
||||
BidirectionalPacketBasedChannel channel(packet2stream);
|
||||
|
||||
StreamToPacketSegmenter stream2packet(channel);
|
||||
|
||||
// now listen for it
|
||||
for (;;) {
|
||||
memset(buf, 0, sizeof(buf));
|
||||
// returns as soon as there is some data
|
||||
ssize_t n_received = recv(sock_fd, buf, sizeof(buf), 0);
|
||||
|
||||
// -1 indicates error and 0 means that the client gracefully terminated
|
||||
if (n_received == -1 || n_received == 0) {
|
||||
close(sock_fd);
|
||||
return n_received;
|
||||
}
|
||||
|
||||
// input processing stack
|
||||
size_t processed = 0;
|
||||
stream2packet.process_bytes(buf, n_received, &processed);
|
||||
}
|
||||
}
|
||||
|
||||
// function to check if a worker thread handling a single client is done
|
||||
template<typename T>
|
||||
bool future_is_ready(std::future<T>& t){
|
||||
return t.wait_for(std::chrono::seconds(0)) == std::future_status::ready;
|
||||
}
|
||||
|
||||
int serve_on_tcp(unsigned int port) {
|
||||
struct sockaddr_in6 si_me, si_other;
|
||||
int s;
|
||||
|
||||
|
||||
if ((s=socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP)) == -1) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
memset((char *) &si_me, 0, sizeof(si_me));
|
||||
si_me.sin6_family = AF_INET6;
|
||||
si_me.sin6_port = htons(port);
|
||||
si_me.sin6_flowinfo = 0;
|
||||
si_me.sin6_addr = in6addr_any;
|
||||
if (bind(s, reinterpret_cast<struct sockaddr *>(&si_me), sizeof(si_me)) == -1) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
listen(s, 128); // make this socket a passive socket
|
||||
std::vector<std::future<int>> serv_pool;
|
||||
for (;;) {
|
||||
memset(&si_other, 0, sizeof(si_other));
|
||||
|
||||
socklen_t silen = sizeof(si_other);
|
||||
// TODO: Add a limit on accepting connections
|
||||
int client_portal_fd = accept(s, reinterpret_cast<sockaddr *>(&si_other), &silen); // blocking call
|
||||
serv_pool.push_back(std::async(std::launch::async, serve_client, client_portal_fd));
|
||||
// do a little clean up on the pool
|
||||
for (std::vector<std::future<int>>::iterator it = serv_pool.end()-1; it >= serv_pool.begin(); --it) {
|
||||
if (future_is_ready(*it)) {
|
||||
// we can erase this thread
|
||||
serv_pool.erase(it);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
close(s);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
|
||||
#include <arpa/inet.h>
|
||||
#include <netinet/in.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/socket.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <fibre/protocol.hpp>
|
||||
|
||||
#define UDP_RX_BUF_LEN 512
|
||||
#define UDP_TX_BUF_LEN 512
|
||||
|
||||
|
||||
class UDPPacketSender : public PacketSink {
|
||||
public:
|
||||
UDPPacketSender(int socket_fd, struct sockaddr_in6 *si_other) :
|
||||
_socket_fd(socket_fd),
|
||||
_si_other(si_other)
|
||||
{}
|
||||
|
||||
size_t get_mtu() { return UDP_TX_BUF_LEN; }
|
||||
|
||||
int process_packet(const uint8_t* buffer, size_t length) {
|
||||
// cannot send partial packets
|
||||
if (length > get_mtu())
|
||||
return -1;
|
||||
|
||||
int status = sendto(_socket_fd, buffer, length, 0, reinterpret_cast<struct sockaddr*>(_si_other), sizeof(*_si_other));
|
||||
return (status == -1) ? -1 : 0;
|
||||
}
|
||||
|
||||
private:
|
||||
int _socket_fd;
|
||||
struct sockaddr_in6 *_si_other;
|
||||
};
|
||||
|
||||
|
||||
|
||||
int serve_on_udp(unsigned int port) {
|
||||
struct sockaddr_in6 si_me, si_other;
|
||||
int s;
|
||||
socklen_t slen = sizeof(si_other);
|
||||
uint8_t buf[UDP_RX_BUF_LEN];
|
||||
|
||||
if ((s=socket(AF_INET6, SOCK_DGRAM, IPPROTO_UDP)) == -1)
|
||||
return -1;
|
||||
|
||||
memset((char *) &si_me, 0, sizeof(si_me));
|
||||
si_me.sin6_family = AF_INET6;
|
||||
si_me.sin6_port = htons(port);
|
||||
si_me.sin6_flowinfo = 0;
|
||||
si_me.sin6_addr= in6addr_any;
|
||||
if (bind(s, reinterpret_cast<struct sockaddr *>(&si_me), sizeof(si_me)) == -1)
|
||||
return -1;
|
||||
|
||||
for (;;) {
|
||||
ssize_t n_received = recvfrom(s, buf, sizeof(buf), 0, reinterpret_cast<struct sockaddr *>(&si_other), &slen);
|
||||
if (n_received == -1)
|
||||
return -1;
|
||||
//printf("Received packet from %s:%d\nData: %s\n\n",
|
||||
// inet_ntoa(si_other.sin_addr), ntohs(si_other.sin_port), buf);
|
||||
|
||||
UDPPacketSender udp_packet_output(s, &si_other);
|
||||
BidirectionalPacketBasedChannel udp_channel(udp_packet_output);
|
||||
udp_channel.process_packet(buf, n_received);
|
||||
}
|
||||
|
||||
close(s);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
|
||||
/* Includes ------------------------------------------------------------------*/
|
||||
|
||||
#include <memory>
|
||||
#include <stdlib.h>
|
||||
|
||||
#include <fibre/protocol.hpp>
|
||||
#include <fibre/crc.hpp>
|
||||
|
||||
/* Private defines -----------------------------------------------------------*/
|
||||
/* Private macros ------------------------------------------------------------*/
|
||||
/* Private typedef -----------------------------------------------------------*/
|
||||
/* Global constant data ------------------------------------------------------*/
|
||||
/* Global variables ----------------------------------------------------------*/
|
||||
|
||||
/* Private constant data -----------------------------------------------------*/
|
||||
/* Private variables ---------------------------------------------------------*/
|
||||
/* Private function prototypes -----------------------------------------------*/
|
||||
|
||||
static void hexdump(const uint8_t* buf, size_t len);
|
||||
|
||||
/* Function implementations --------------------------------------------------*/
|
||||
|
||||
#if 0
|
||||
void hexdump(const uint8_t* buf, size_t len) {
|
||||
for (size_t pos = 0; pos < len; ++pos) {
|
||||
printf(" %02x", buf[pos]);
|
||||
if ((((pos + 1) % 16) == 0) || ((pos + 1) == len))
|
||||
printf("\r\n");
|
||||
osDelay(2);
|
||||
}
|
||||
}
|
||||
#else
|
||||
void hexdump(const uint8_t* buf, size_t len) {
|
||||
(void) buf;
|
||||
(void) len;
|
||||
}
|
||||
#endif
|
||||
|
||||
|
||||
|
||||
int StreamToPacketSegmenter::process_bytes(const uint8_t *buffer, size_t length, size_t* processed_bytes) {
|
||||
int result = 0;
|
||||
|
||||
while (length--) {
|
||||
if (header_index_ < sizeof(header_buffer_)) {
|
||||
// Process header byte
|
||||
header_buffer_[header_index_++] = *buffer;
|
||||
if (header_index_ == 1 && header_buffer_[0] != CANONICAL_PREFIX) {
|
||||
header_index_ = 0;
|
||||
} else if (header_index_ == 2 && (header_buffer_[1] & 0x80)) {
|
||||
header_index_ = 0; // TODO: support packets larger than 128 bytes
|
||||
} else if (header_index_ == 3 && calc_crc8<CANONICAL_CRC8_POLYNOMIAL>(CANONICAL_CRC8_INIT, header_buffer_, 3)) {
|
||||
header_index_ = 0;
|
||||
} else if (header_index_ == 3) {
|
||||
packet_length_ = header_buffer_[1] + 2;
|
||||
}
|
||||
} else if (packet_index_ < sizeof(packet_buffer_)) {
|
||||
// Process payload byte
|
||||
packet_buffer_[packet_index_++] = *buffer;
|
||||
}
|
||||
|
||||
// If both header and packet are fully received, hand it on to the packet processor
|
||||
if (header_index_ == 3 && packet_index_ == packet_length_) {
|
||||
if (calc_crc16<CANONICAL_CRC16_POLYNOMIAL>(CANONICAL_CRC16_INIT, packet_buffer_, packet_length_) == 0) {
|
||||
result |= output_.process_packet(packet_buffer_, packet_length_ - 2);
|
||||
}
|
||||
header_index_ = packet_index_ = packet_length_ = 0;
|
||||
}
|
||||
buffer++;
|
||||
if (processed_bytes)
|
||||
(*processed_bytes)++;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
int StreamBasedPacketSink::process_packet(const uint8_t *buffer, size_t length) {
|
||||
// TODO: support buffer size >= 128
|
||||
if (length >= 128)
|
||||
return -1;
|
||||
|
||||
LOG_FIBRE("send header\r\n");
|
||||
uint8_t header[] = {
|
||||
CANONICAL_PREFIX,
|
||||
static_cast<uint8_t>(length),
|
||||
0
|
||||
};
|
||||
header[2] = calc_crc8<CANONICAL_CRC8_POLYNOMIAL>(CANONICAL_CRC8_INIT, header, 2);
|
||||
|
||||
if (output_.process_bytes(header, sizeof(header), nullptr))
|
||||
return -1;
|
||||
LOG_FIBRE("send payload:\r\n");
|
||||
hexdump(buffer, length);
|
||||
if (output_.process_bytes(buffer, length, nullptr))
|
||||
return -1;
|
||||
|
||||
LOG_FIBRE("send crc16\r\n");
|
||||
uint16_t crc16 = calc_crc16<CANONICAL_CRC16_POLYNOMIAL>(CANONICAL_CRC16_INIT, buffer, length);
|
||||
uint8_t crc16_buffer[] = {
|
||||
(uint8_t)((crc16 >> 8) & 0xff),
|
||||
(uint8_t)((crc16 >> 0) & 0xff)
|
||||
};
|
||||
if (output_.process_bytes(crc16_buffer, 2, nullptr))
|
||||
return -1;
|
||||
LOG_FIBRE("sent!\r\n");
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
// Returns part of the JSON interface definition.
|
||||
bool fibre::endpoint0_handler(fibre::cbufptr_t* input_buffer, fibre::bufptr_t* output_buffer) {
|
||||
// The request must contain a 32 bit integer to specify an offset
|
||||
std::optional<uint32_t> offset = read_le<uint32_t>(input_buffer);
|
||||
|
||||
if (!offset.has_value()) {
|
||||
// Didn't receive any offset
|
||||
return false;
|
||||
} else if (offset.value() == 0xffffffff) {
|
||||
// If the offset is special value 0xFFFFFFFF, send back the JSON version ID instead
|
||||
return write_le<uint32_t>(json_version_id_, output_buffer);
|
||||
} else if (offset.value() >= embedded_json_length) {
|
||||
// Attempt to read beyond the buffer end - return empty response
|
||||
return true;
|
||||
} else {
|
||||
// Return part of the json file
|
||||
size_t n_copy = std::min(output_buffer->size(), embedded_json_length - (size_t)offset.value());
|
||||
memcpy(output_buffer->begin(), embedded_json + offset.value(), n_copy);
|
||||
*output_buffer = output_buffer->skip(n_copy);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
int BidirectionalPacketBasedChannel::process_packet(const uint8_t* buffer, size_t length) {
|
||||
LOG_FIBRE("got packet of length %d: \r\n", length);
|
||||
hexdump(buffer, length);
|
||||
if (length < 4)
|
||||
return -1;
|
||||
|
||||
uint16_t seq_no = read_le<uint16_t>(&buffer, &length);
|
||||
|
||||
if (seq_no & 0x8000) {
|
||||
// TODO: ack handling
|
||||
} else {
|
||||
// TODO: think about some kind of ordering guarantees
|
||||
// currently the seq_no is just used to associate a response with a request
|
||||
|
||||
uint16_t endpoint_id = read_le<uint16_t>(&buffer, &length);
|
||||
bool expect_response = endpoint_id & 0x8000;
|
||||
endpoint_id &= 0x7fff;
|
||||
|
||||
// Verify packet trailer. The expected trailer value depends on the selected endpoint.
|
||||
// For endpoint 0 this is just the protocol version, for all other endpoints it's a
|
||||
// CRC over the entire JSON descriptor tree (this may change in future versions).
|
||||
uint16_t expected_trailer = endpoint_id ? fibre::json_crc_ : PROTOCOL_VERSION;
|
||||
uint16_t actual_trailer = buffer[length - 2] | (buffer[length - 1] << 8);
|
||||
if (expected_trailer != actual_trailer) {
|
||||
LOG_FIBRE("trailer mismatch for endpoint %d: expected %04x, got %04x\r\n", endpoint_id, expected_trailer, actual_trailer);
|
||||
return -1;
|
||||
}
|
||||
LOG_FIBRE("trailer ok for endpoint %d\r\n", endpoint_id);
|
||||
|
||||
// TODO: if more bytes than the MTU were requested, should we abort or just return as much as possible?
|
||||
|
||||
uint16_t expected_response_length = read_le<uint16_t>(&buffer, &length);
|
||||
|
||||
// Limit response length according to our local TX buffer size
|
||||
if (expected_response_length > sizeof(tx_buf_) - 2)
|
||||
expected_response_length = sizeof(tx_buf_) - 2;
|
||||
|
||||
fibre::cbufptr_t input_buffer{buffer, length - 2};
|
||||
fibre::bufptr_t output_buffer{tx_buf_ + 2, expected_response_length};
|
||||
fibre::endpoint_handler(endpoint_id, &input_buffer, &output_buffer);
|
||||
|
||||
// Send response
|
||||
if (expect_response) {
|
||||
size_t actual_response_length = expected_response_length - output_buffer.size() + 2;
|
||||
write_le<uint16_t>(seq_no | 0x8000, tx_buf_);
|
||||
|
||||
LOG_FIBRE("send packet:\r\n");
|
||||
hexdump(tx_buf_, actual_response_length);
|
||||
output_.process_packet(tx_buf_, actual_response_length);
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
/*[# This is the original template, thus the warning below does not apply to this file #]
|
||||
* ============================ WARNING ============================
|
||||
* ==== This is an autogenerated file. ====
|
||||
* ==== Any changes to this file will be lost when recompiling. ====
|
||||
* =================================================================
|
||||
*
|
||||
* This file contains support functions for the ODrive ASCII protocol.
|
||||
*
|
||||
* TODO: might generalize this as an approach to runtime introspection.
|
||||
*/
|
||||
|
||||
#include <fibre/introspection.hpp>
|
||||
|
||||
#pragma GCC push_options
|
||||
#pragma GCC optimize ("s")
|
||||
|
||||
[% for intf in interfaces.values() %][% if not intf.builtin %]
|
||||
template<typename T>
|
||||
struct [[intf.fullname | to_pascal_case]]TypeInfo : TypeInfo {
|
||||
using TypeInfo::TypeInfo;
|
||||
static const PropertyInfo property_table[];
|
||||
static const [[intf.fullname | to_pascal_case]]TypeInfo<T> singleton;
|
||||
static Introspectable make_introspectable(T& obj) { return TypeInfo::make_introspectable(&obj, &singleton); }
|
||||
|
||||
introspectable_storage_t get_child(introspectable_storage_t obj, size_t idx) const override {
|
||||
T* ptr = *(T**)&obj;
|
||||
introspectable_storage_t res;
|
||||
switch (idx) {
|
||||
[%- for property in intf.attributes.values() %]
|
||||
case [[loop.index0]]: *(decltype([[intf.c_name]]::get_[[property.name]](std::declval<T*>()))*)(&res) = [[intf.c_name]]::get_[[property.name]](ptr); break;
|
||||
[%- endfor %]
|
||||
}
|
||||
return res;
|
||||
}
|
||||
};
|
||||
[% endif %][% endfor %]
|
||||
|
||||
[% for intf in interfaces.values() %][% if not intf.builtin %]
|
||||
template<typename T>
|
||||
const PropertyInfo [[intf.fullname | to_pascal_case]]TypeInfo<T>::property_table[] = {
|
||||
[%- for property in intf.attributes.values() %]
|
||||
{"[[property.name]]", &[[(property.type.purename or property.type.fullname) | to_pascal_case]]TypeInfo<std::remove_reference_t<decltype(*[[intf.c_name]]::get_[[property.name]](std::declval<T*>()))>>::singleton},
|
||||
[%- endfor %]
|
||||
};
|
||||
template<typename T>
|
||||
const [[intf.fullname | to_pascal_case]]TypeInfo<T> [[intf.fullname | to_pascal_case]]TypeInfo<T>::singleton{[[intf.fullname | to_pascal_case]]TypeInfo<T>::property_table, sizeof([[intf.fullname | to_pascal_case]]TypeInfo<T>::property_table) / sizeof([[intf.fullname | to_pascal_case]]TypeInfo<T>::property_table[0])};
|
||||
|
||||
[% endif %][% endfor %]
|
||||
|
||||
#pragma GCC pop_options
|
||||
@@ -0,0 +1,16 @@
|
||||
|
||||
# Python Distribution / packaging
|
||||
.Python
|
||||
/dist/
|
||||
/*.egg-info/
|
||||
/MANIFEST
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
@@ -0,0 +1,5 @@
|
||||
|
||||
from .discovery import find_any, find_all
|
||||
from .utils import Event, Logger, TimeoutError
|
||||
from .protocol import ChannelBrokenException, ChannelDamagedException
|
||||
from .shell import launch_shell
|
||||
@@ -0,0 +1,179 @@
|
||||
"""
|
||||
Provides functions for the discovery of Fibre nodes
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
import threading
|
||||
import traceback
|
||||
import struct
|
||||
import fibre.protocol
|
||||
import fibre.utils
|
||||
import fibre.remote_object
|
||||
from fibre.utils import Event, Logger
|
||||
from fibre.protocol import ChannelBrokenException, TimeoutError
|
||||
import appdirs
|
||||
import os
|
||||
|
||||
# Load all installed transport layers
|
||||
|
||||
channel_types = {}
|
||||
|
||||
try:
|
||||
import fibre.usbbulk_transport
|
||||
channel_types['usb'] = fibre.usbbulk_transport.discover_channels
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
import fibre.serial_transport
|
||||
channel_types['serial'] = fibre.serial_transport.discover_channels
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
import fibre.tcp_transport
|
||||
channel_types['tcp'] = fibre.tcp_transport.discover_channels
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
import fibre.udp_transport
|
||||
channel_types['udp'] = fibre.udp_transport.discover_channels
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
def noprint(text):
|
||||
pass
|
||||
|
||||
def find_all(path, serial_number,
|
||||
did_discover_object_callback,
|
||||
search_cancellation_token,
|
||||
channel_termination_token,
|
||||
logger):
|
||||
"""
|
||||
Starts scanning for Fibre nodes that match the specified path spec and calls
|
||||
the callback for each Fibre node that is found.
|
||||
This function is non-blocking.
|
||||
"""
|
||||
|
||||
def did_discover_channel(channel):
|
||||
"""
|
||||
Inits an object from a given channel and then calls did_discover_object_callback
|
||||
with the created object
|
||||
This queries the endpoint 0 on that channel to gain information
|
||||
about the interface, which is then used to init the corresponding object.
|
||||
"""
|
||||
try:
|
||||
logger.debug("Connecting to device on " + channel._name)
|
||||
|
||||
cache_dir = appdirs.user_cache_dir("odrivetool")
|
||||
cache_path = None
|
||||
|
||||
# Fetch the json version tag to check cache (only supported on firmware v0.5 or later)
|
||||
try:
|
||||
json_version_tag = channel.remote_endpoint_operation(0, struct.pack("<I", 0xffffffff), True, 4)
|
||||
json_version_tag = struct.unpack("<I", json_version_tag)[0]
|
||||
|
||||
logger.debug("Device reported JSON version ID: {:08d}".format(json_version_tag))
|
||||
cache_path = os.path.join(cache_dir, 'fibre_schema_cache_{:08d}'.format(json_version_tag))
|
||||
except:
|
||||
logger.debug("Failed to get JSON checksum")
|
||||
|
||||
# Check cache
|
||||
json_data = None
|
||||
try:
|
||||
if not cache_path is None:
|
||||
with open(cache_path, 'rb') as fp:
|
||||
json_crc16 = fibre.protocol.calc_crc16(fibre.protocol.PROTOCOL_VERSION, fp.read())
|
||||
fp.seek(0)
|
||||
json_data = json.load(fp)
|
||||
except:
|
||||
logger.debug("Failed load JSON cache file {}".format(cache_path))
|
||||
|
||||
# Fallback to loading JSON from device
|
||||
if json_data is None:
|
||||
# Downloading json data
|
||||
logger.info("Downloading json data from ODrive... (this might take a while)")
|
||||
json_bytes = channel.remote_endpoint_read_buffer(0)
|
||||
try:
|
||||
json_string = json_bytes.decode("ascii")
|
||||
except UnicodeDecodeError:
|
||||
logger.debug("Device responded on endpoint 0 with something that is not ASCII")
|
||||
raise UnicodeDecodeError
|
||||
|
||||
json_crc16 = fibre.protocol.calc_crc16(fibre.protocol.PROTOCOL_VERSION, json_bytes)
|
||||
json_data = json.loads(json_string)
|
||||
|
||||
# Save JSON to cache
|
||||
if not cache_path is None:
|
||||
logger.debug("Creating new JSON cache file {}".format(cache_path))
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
with open(cache_path, 'w+') as json_cache:
|
||||
json_cache.write(json_string)
|
||||
logger.debug("Saved JSON to cache file {}".format(cache_path))
|
||||
|
||||
channel._interface_definition_crc = json_crc16
|
||||
|
||||
logger.debug("JSON: " + str(json_data).replace("{'name'", "\n{'name'"))
|
||||
|
||||
json_data = {"name": "fibre_node", "members": json_data}
|
||||
obj = fibre.remote_object.RemoteObject(json_data, None, channel, logger)
|
||||
|
||||
obj.__dict__['_json_data'] = json_data['members']
|
||||
obj.__dict__['_json_crc'] = json_crc16
|
||||
|
||||
device_serial_number = fibre.utils.get_serial_number_str(obj)
|
||||
if serial_number != None and device_serial_number != serial_number:
|
||||
logger.debug("Ignoring device with serial number {}".format(device_serial_number))
|
||||
return
|
||||
|
||||
did_discover_object_callback(obj)
|
||||
|
||||
|
||||
except Exception:
|
||||
logger.debug("Unexpected exception after discovering channel: " + traceback.format_exc())
|
||||
|
||||
# For each connection type, kick off an appropriate discovery loop
|
||||
for search_spec in path.split(','):
|
||||
prefix = search_spec.split(':')[0]
|
||||
the_rest = ':'.join(search_spec.split(':')[1:])
|
||||
if prefix in channel_types:
|
||||
t = threading.Thread(target=channel_types[prefix],
|
||||
args=(the_rest, serial_number, did_discover_channel, search_cancellation_token, channel_termination_token, logger))
|
||||
t.daemon = True
|
||||
t.start()
|
||||
else:
|
||||
raise Exception("Invalid path spec \"{}\"".format(search_spec))
|
||||
|
||||
|
||||
def find_any(path="usb", serial_number=None,
|
||||
search_cancellation_token=None, channel_termination_token=None,
|
||||
timeout=None, logger=Logger(verbose=False), find_multiple=False):
|
||||
"""
|
||||
Blocks until the first matching Fibre node is connected and then returns that node
|
||||
"""
|
||||
result = []
|
||||
done_signal = Event(search_cancellation_token)
|
||||
def did_discover_object(obj):
|
||||
result.append(obj)
|
||||
if find_multiple:
|
||||
if len(result) >= int(find_multiple):
|
||||
done_signal.set()
|
||||
else:
|
||||
done_signal.set()
|
||||
|
||||
find_all(path, serial_number, did_discover_object, done_signal, channel_termination_token, logger)
|
||||
try:
|
||||
done_signal.wait(timeout=timeout)
|
||||
except TimeoutError:
|
||||
if not find_multiple:
|
||||
return None
|
||||
finally:
|
||||
done_signal.set() # terminate find_all
|
||||
|
||||
if find_multiple:
|
||||
return result
|
||||
else:
|
||||
return result[0]
|
||||
@@ -0,0 +1,359 @@
|
||||
# See protocol.hpp for an overview of the protocol
|
||||
|
||||
import time
|
||||
import struct
|
||||
import sys
|
||||
import threading
|
||||
import traceback
|
||||
#import fibre.utils
|
||||
from fibre.utils import Event, wait_any, TimeoutError
|
||||
|
||||
import abc
|
||||
if sys.version_info >= (3, 4):
|
||||
ABC = abc.ABC
|
||||
else:
|
||||
ABC = abc.ABCMeta('ABC', (), {})
|
||||
|
||||
if sys.version_info < (3, 3):
|
||||
from monotonic import monotonic
|
||||
time.monotonic = monotonic
|
||||
|
||||
SYNC_BYTE = 0xAA
|
||||
CRC8_INIT = 0x42
|
||||
CRC16_INIT = 0x1337
|
||||
PROTOCOL_VERSION = 1
|
||||
|
||||
CRC8_DEFAULT = 0x37 # this must match the polynomial in the C++ implementation
|
||||
CRC16_DEFAULT = 0x3d65 # this must match the polynomial in the C++ implementation
|
||||
|
||||
MAX_PACKET_SIZE = 128
|
||||
|
||||
# For more information on the CRC algorithm refer to protocol.md
|
||||
|
||||
def calc_crc(remainder, value, polynomial, bitwidth):
|
||||
topbit = (1 << (bitwidth - 1))
|
||||
|
||||
# Bring the next byte into the remainder.
|
||||
remainder ^= (value << (bitwidth - 8))
|
||||
for bitnumber in range(0,8):
|
||||
if (remainder & topbit):
|
||||
remainder = (remainder << 1) ^ polynomial
|
||||
else:
|
||||
remainder = (remainder << 1)
|
||||
|
||||
return remainder & ((1 << bitwidth) - 1)
|
||||
|
||||
def calc_crc8(remainder, value):
|
||||
if isinstance(value, bytearray) or isinstance(value, bytes) or isinstance(value, list):
|
||||
for byte in value:
|
||||
if not isinstance(byte,int):
|
||||
byte = ord(byte)
|
||||
remainder = calc_crc(remainder, byte, CRC8_DEFAULT, 8)
|
||||
else:
|
||||
remainder = calc_crc(remainder, byte, CRC8_DEFAULT, 8)
|
||||
return remainder
|
||||
|
||||
def calc_crc16(remainder, value):
|
||||
if isinstance(value, bytearray) or isinstance(value, bytes) or isinstance(value, list):
|
||||
for byte in value:
|
||||
if not isinstance(byte, int):
|
||||
byte = ord(byte)
|
||||
remainder = calc_crc(remainder, byte, CRC16_DEFAULT, 16)
|
||||
else:
|
||||
remainder = calc_crc(remainder, value, CRC16_DEFAULT, 16)
|
||||
return remainder
|
||||
|
||||
|
||||
class DeviceInitException(Exception):
|
||||
pass
|
||||
|
||||
class ChannelDamagedException(Exception):
|
||||
"""
|
||||
Raised when the channel is temporarily broken and a
|
||||
resend of the message might be successful
|
||||
"""
|
||||
pass
|
||||
|
||||
class ChannelBrokenException(Exception):
|
||||
"""
|
||||
Raised when the channel is permanently broken
|
||||
"""
|
||||
pass
|
||||
|
||||
|
||||
class StreamSource(ABC):
|
||||
@abc.abstractmethod
|
||||
def get_bytes(self, n_bytes, deadline):
|
||||
pass
|
||||
|
||||
class StreamSink(ABC):
|
||||
@abc.abstractmethod
|
||||
def process_bytes(self, bytes):
|
||||
pass
|
||||
|
||||
class PacketSource(ABC):
|
||||
@abc.abstractmethod
|
||||
def get_packet(self, deadline):
|
||||
pass
|
||||
|
||||
class PacketSink(ABC):
|
||||
@abc.abstractmethod
|
||||
def process_packet(self, packet):
|
||||
pass
|
||||
|
||||
|
||||
class StreamToPacketSegmenter(StreamSink):
|
||||
def __init__(self, output):
|
||||
self._header = []
|
||||
self._packet = []
|
||||
self._packet_length = 0
|
||||
self._output = output
|
||||
|
||||
def process_bytes(self, bytes):
|
||||
"""
|
||||
Processes an arbitrary number of bytes. If one or more full packets are
|
||||
are received, they are sent to this instance's output PacketSink.
|
||||
Incomplete packets are buffered between subsequent calls to this function.
|
||||
"""
|
||||
|
||||
for byte in bytes:
|
||||
if (len(self._header) < 3):
|
||||
# Process header byte
|
||||
self._header.append(byte)
|
||||
if (len(self._header) == 1) and (self._header[0] != SYNC_BYTE):
|
||||
self._header = []
|
||||
elif (len(self._header) == 2) and (self._header[1] & 0x80):
|
||||
self._header = [] # TODO: support packets larger than 128 bytes
|
||||
elif (len(self._header) == 3) and calc_crc8(CRC8_INIT, self._header):
|
||||
self._header = []
|
||||
elif (len(self._header) == 3):
|
||||
self._packet_length = self._header[1] + 2
|
||||
else:
|
||||
# Process payload byte
|
||||
self._packet.append(byte)
|
||||
|
||||
# If both header and packet are fully received, hand it on to the packet processor
|
||||
if (len(self._header) == 3) and (len(self._packet) == self._packet_length):
|
||||
if calc_crc16(CRC16_INIT, self._packet) == 0:
|
||||
self._output.process_packet(self._packet[:-2])
|
||||
self._header = []
|
||||
self._packet = []
|
||||
self._packet_length = 0
|
||||
|
||||
|
||||
class StreamBasedPacketSink(PacketSink):
|
||||
def __init__(self, output):
|
||||
self._output = output
|
||||
|
||||
def process_packet(self, packet):
|
||||
if (len(packet) >= MAX_PACKET_SIZE):
|
||||
raise NotImplementedError("packet larger than 127 currently not supported")
|
||||
|
||||
header = bytearray()
|
||||
header.append(SYNC_BYTE)
|
||||
header.append(len(packet))
|
||||
header.append(calc_crc8(CRC8_INIT, header))
|
||||
|
||||
self._output.process_bytes(header)
|
||||
self._output.process_bytes(packet)
|
||||
|
||||
# append CRC in big endian
|
||||
crc16 = calc_crc16(CRC16_INIT, packet)
|
||||
self._output.process_bytes(struct.pack('>H', crc16))
|
||||
|
||||
class PacketFromStreamConverter(PacketSource):
|
||||
def __init__(self, input):
|
||||
self._input = input
|
||||
|
||||
def get_packet(self, deadline):
|
||||
"""
|
||||
Requests bytes from the underlying input stream until a full packet is
|
||||
received or the deadline is reached, in which case None is returned. A
|
||||
deadline before the current time corresponds to non-blocking mode.
|
||||
"""
|
||||
while True:
|
||||
header = bytes()
|
||||
|
||||
# TODO: sometimes this call hangs, even though the device apparently sent something
|
||||
header = header + self._input.get_bytes_or_fail(1, deadline)
|
||||
if (header[0] != SYNC_BYTE):
|
||||
#print("sync byte mismatch")
|
||||
continue
|
||||
|
||||
header = header + self._input.get_bytes_or_fail(1, deadline)
|
||||
if (header[1] & 0x80):
|
||||
#print("packet too large")
|
||||
continue # TODO: support packets larger than 128 bytes
|
||||
|
||||
header = header + self._input.get_bytes_or_fail(1, deadline)
|
||||
if calc_crc8(CRC8_INIT, header) != 0:
|
||||
#print("crc8 mismatch")
|
||||
continue
|
||||
|
||||
packet_length = header[1] + 2
|
||||
#print("wait for {} bytes".format(packet_length))
|
||||
packet = self._input.get_bytes_or_fail(packet_length, deadline)
|
||||
if calc_crc16(CRC16_INIT, packet) != 0:
|
||||
#print("crc16 mismatch")
|
||||
continue
|
||||
return packet[:-2]
|
||||
|
||||
|
||||
class Channel(PacketSink):
|
||||
# Choose these parameters to be sensible for a specific transport layer
|
||||
_resend_timeout = 5.0 # [s]
|
||||
_send_attempts = 5
|
||||
|
||||
def __init__(self, name, input, output, cancellation_token, logger):
|
||||
"""
|
||||
Params:
|
||||
input: A PacketSource where this channel will source packets from on
|
||||
demand. Alternatively packets can be provided to this channel
|
||||
directly by calling process_packet on this instance.
|
||||
output: A PacketSink where this channel will put outgoing packets.
|
||||
"""
|
||||
self._name = name
|
||||
self._input = input
|
||||
self._output = output
|
||||
self._logger = logger
|
||||
self._outbound_seq_no = 0
|
||||
self._interface_definition_crc = 0
|
||||
self._expected_acks = {}
|
||||
self._responses = {}
|
||||
self._my_lock = threading.Lock()
|
||||
self._channel_broken = Event(cancellation_token)
|
||||
self.start_receiver_thread(Event(self._channel_broken))
|
||||
|
||||
def start_receiver_thread(self, cancellation_token):
|
||||
"""
|
||||
Starts the receiver thread that processes incoming messages.
|
||||
The thread quits as soon as the channel enters a broken state.
|
||||
"""
|
||||
def receiver_thread():
|
||||
error_ctr = 0
|
||||
try:
|
||||
while (not cancellation_token.is_set() and not self._channel_broken.is_set()
|
||||
and error_ctr < 10):
|
||||
# Set an arbitrary deadline because the get_packet function
|
||||
# currently doesn't support a cancellation_token
|
||||
deadline = time.monotonic() + 1.0
|
||||
try:
|
||||
response = self._input.get_packet(deadline)
|
||||
except TimeoutError:
|
||||
continue # try again
|
||||
except ChannelDamagedException:
|
||||
error_ctr += 1
|
||||
continue # try again
|
||||
if (error_ctr > 0):
|
||||
error_ctr -= 1
|
||||
# Process response
|
||||
# This should not throw an exception, otherwise the channel breaks
|
||||
self.process_packet(response)
|
||||
#print("receiver thread is exiting")
|
||||
except Exception:
|
||||
self._logger.debug("receiver thread is exiting: " + traceback.format_exc())
|
||||
finally:
|
||||
self._channel_broken.set()
|
||||
t = threading.Thread(target=receiver_thread)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
def remote_endpoint_operation(self, endpoint_id, input, expect_ack, output_length):
|
||||
if input is None:
|
||||
input = bytearray(0)
|
||||
if (len(input) >= 128):
|
||||
raise Exception("packet larger than 127 currently not supported")
|
||||
|
||||
if (expect_ack):
|
||||
endpoint_id |= 0x8000
|
||||
|
||||
self._my_lock.acquire()
|
||||
try:
|
||||
self._outbound_seq_no = ((self._outbound_seq_no + 1) & 0x7fff)
|
||||
seq_no = self._outbound_seq_no
|
||||
finally:
|
||||
self._my_lock.release()
|
||||
seq_no |= 0x80 # FIXME: we hardwire one bit of the seq-no to 1 to avoid conflicts with the ascii protocol
|
||||
packet = struct.pack('<HHH', seq_no, endpoint_id, output_length)
|
||||
packet = packet + input
|
||||
|
||||
crc16 = calc_crc16(CRC16_INIT, packet)
|
||||
if (endpoint_id & 0x7fff == 0):
|
||||
trailer = PROTOCOL_VERSION
|
||||
else:
|
||||
trailer = self._interface_definition_crc
|
||||
#print("append trailer " + trailer)
|
||||
packet = packet + struct.pack('<H', trailer)
|
||||
|
||||
if (expect_ack):
|
||||
ack_event = Event()
|
||||
self._expected_acks[seq_no] = ack_event
|
||||
try:
|
||||
attempt = 0
|
||||
while (attempt < self._send_attempts):
|
||||
self._my_lock.acquire()
|
||||
try:
|
||||
self._output.process_packet(packet)
|
||||
except ChannelDamagedException:
|
||||
attempt += 1
|
||||
continue # resend
|
||||
except TimeoutError:
|
||||
attempt += 1
|
||||
continue # resend
|
||||
finally:
|
||||
self._my_lock.release()
|
||||
# Wait for ACK until the resend timeout is exceeded
|
||||
try:
|
||||
if wait_any(self._resend_timeout, ack_event, self._channel_broken) != 0:
|
||||
raise ChannelBrokenException()
|
||||
except TimeoutError:
|
||||
attempt += 1
|
||||
continue # resend
|
||||
return self._responses.pop(seq_no)
|
||||
# TODO: record channel statistics
|
||||
raise ChannelBrokenException() # Too many resend attempts
|
||||
finally:
|
||||
self._expected_acks.pop(seq_no)
|
||||
self._responses.pop(seq_no, None)
|
||||
else:
|
||||
# fire and forget
|
||||
self._output.process_packet(packet)
|
||||
return None
|
||||
|
||||
def remote_endpoint_read_buffer(self, endpoint_id):
|
||||
"""
|
||||
Handles reads from long endpoints
|
||||
"""
|
||||
# TODO: handle device that could (maliciously) send infinite stream
|
||||
buffer = bytes()
|
||||
while True:
|
||||
chunk_length = 512
|
||||
chunk = self.remote_endpoint_operation(endpoint_id, struct.pack("<I", len(buffer)), True, chunk_length)
|
||||
if (len(chunk) == 0):
|
||||
break
|
||||
buffer += chunk
|
||||
return buffer
|
||||
|
||||
def process_packet(self, packet):
|
||||
#print("process packet")
|
||||
packet = bytes(packet)
|
||||
if (len(packet) < 2):
|
||||
raise Exception("packet too short")
|
||||
|
||||
seq_no = struct.unpack('<H', packet[0:2])[0]
|
||||
|
||||
if (seq_no & 0x8000):
|
||||
seq_no &= 0x7fff
|
||||
ack_signal = self._expected_acks.get(seq_no, None)
|
||||
if (ack_signal):
|
||||
self._responses[seq_no] = packet[2:]
|
||||
ack_signal.set()
|
||||
#print("received ack for packet " + str(seq_no))
|
||||
else:
|
||||
print("received unexpected ACK: " + str(seq_no))
|
||||
|
||||
else:
|
||||
#if (calc_crc16(CRC16_INIT, struct.pack('<HBB', PROTOCOL_VERSION, packet[-2], packet[-1]))):
|
||||
# raise Exception("CRC16 mismatch")
|
||||
print("endpoint requested")
|
||||
# TODO: handle local endpoint operation
|
||||
@@ -0,0 +1,264 @@
|
||||
"""
|
||||
Provides functions for the discovery of Fibre nodes
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import struct
|
||||
import threading
|
||||
import fibre.protocol
|
||||
|
||||
class ObjectDefinitionError(Exception):
|
||||
pass
|
||||
|
||||
codecs = {}
|
||||
|
||||
class StructCodec():
|
||||
"""
|
||||
Generic serializer/deserializer based on struct pack
|
||||
"""
|
||||
def __init__(self, struct_format, target_type):
|
||||
self._struct_format = struct_format
|
||||
self._target_type = target_type
|
||||
def get_length(self):
|
||||
return struct.calcsize(self._struct_format)
|
||||
def serialize(self, value):
|
||||
value = self._target_type(value)
|
||||
return struct.pack(self._struct_format, value)
|
||||
def deserialize(self, buffer):
|
||||
value = struct.unpack(self._struct_format, buffer)
|
||||
value = value[0] if len(value) == 1 else value
|
||||
return self._target_type(value)
|
||||
|
||||
class RemoteProperty():
|
||||
"""
|
||||
Used internally by dynamically created objects to translate
|
||||
property assignments and fetches into endpoint operations on the
|
||||
object's associated channel
|
||||
"""
|
||||
def __init__(self, json_data, parent):
|
||||
self._parent = parent
|
||||
self.__channel__ = parent.__channel__
|
||||
id_str = json_data.get("id", None)
|
||||
if id_str is None:
|
||||
raise ObjectDefinitionError("unspecified endpoint ID")
|
||||
self._id = int(id_str)
|
||||
|
||||
self._name = json_data.get("name", None)
|
||||
if self._name is None:
|
||||
self._name = "[anonymous]"
|
||||
|
||||
type_str = json_data.get("type", None)
|
||||
if type_str is None:
|
||||
raise ObjectDefinitionError("unspecified type")
|
||||
|
||||
# Find all codecs that match the type_str and build a dictionary
|
||||
# of the form {type1: codec1, type2: codec2}
|
||||
eligible_types = {k: v[type_str] for (k,v) in codecs.items() if type_str in v}
|
||||
|
||||
if not eligible_types:
|
||||
raise ObjectDefinitionError("unsupported codec {}".format(type_str))
|
||||
|
||||
# TODO: better heuristics to select a matching type (i.e. prefer non lossless)
|
||||
eligible_types = list(eligible_types.items())
|
||||
self._property_type = eligible_types[0][0]
|
||||
self._codec = eligible_types[0][1]
|
||||
|
||||
access_mode = json_data.get("access", "r")
|
||||
self._can_read = 'r' in access_mode
|
||||
self._can_write = 'w' in access_mode
|
||||
|
||||
def get_value(self):
|
||||
buffer = self._parent.__channel__.remote_endpoint_operation(self._id, None, True, self._codec.get_length())
|
||||
return self._codec.deserialize(buffer)
|
||||
|
||||
def set_value(self, value):
|
||||
buffer = self._codec.serialize(value)
|
||||
# TODO: Currenly we wait for an ack here. Settle on the default guarantee.
|
||||
self._parent.__channel__.remote_endpoint_operation(self._id, buffer, True, 0)
|
||||
|
||||
def _dump(self):
|
||||
if self._name == "serial_number":
|
||||
# special case: serial number should be displayed in hex (TODO: generalize)
|
||||
val_str = "{:012X}".format(self.get_value())
|
||||
elif self._name == "error":
|
||||
# special case: errors should be displayed in hex (TODO: generalize)
|
||||
val_str = "0x{:04X}".format(self.get_value())
|
||||
else:
|
||||
val_str = str(self.get_value())
|
||||
return "{} = {} ({})".format(self._name, val_str, self._property_type.__name__)
|
||||
|
||||
class EndpointRefCodec():
|
||||
"""
|
||||
Serializer/deserializer for an endpoint reference
|
||||
"""
|
||||
def get_length(self):
|
||||
return struct.calcsize("<HH")
|
||||
def serialize(self, value):
|
||||
if value is None:
|
||||
(ep_id, ep_crc) = (0, 0)
|
||||
elif isinstance(value, RemoteProperty):
|
||||
(ep_id, ep_crc) = (value._id, value.__channel__._interface_definition_crc)
|
||||
else:
|
||||
raise TypeError("Expected value of type RemoteProperty or None but got '{}'. En example for a RemoteProperty is this expression: odrv0.axis0.controller._remote_attributes['pos_setpoint']".format(type(value).__name__))
|
||||
return struct.pack("<HH", ep_id, ep_crc)
|
||||
def deserialize(self, buffer):
|
||||
return struct.unpack("<HH", buffer)
|
||||
|
||||
codecs[int] = {
|
||||
'int8': StructCodec("<b", int),
|
||||
'uint8': StructCodec("<B", int),
|
||||
'int16': StructCodec("<h", int),
|
||||
'uint16': StructCodec("<H", int),
|
||||
'int32': StructCodec("<i", int),
|
||||
'uint32': StructCodec("<I", int),
|
||||
'int64': StructCodec("<q", int),
|
||||
'uint64': StructCodec("<Q", int)
|
||||
}
|
||||
|
||||
codecs[bool] = {
|
||||
'bool': StructCodec("<?", bool)
|
||||
}
|
||||
|
||||
codecs[float] = {
|
||||
'float': StructCodec("<f", float)
|
||||
}
|
||||
|
||||
codecs[RemoteProperty] = {
|
||||
'endpoint_ref': EndpointRefCodec()
|
||||
}
|
||||
|
||||
|
||||
class RemoteFunction(object):
|
||||
"""
|
||||
Represents a callable function that maps to a function call on a remote object
|
||||
"""
|
||||
def __init__(self, json_data, parent):
|
||||
self._parent = parent
|
||||
id_str = json_data.get("id", None)
|
||||
if id_str is None:
|
||||
raise ObjectDefinitionError("unspecified endpoint ID")
|
||||
self._trigger_id = int(id_str)
|
||||
|
||||
self._name = json_data.get("name", None)
|
||||
if self._name is None:
|
||||
self._name = "[anonymous]"
|
||||
|
||||
self._inputs = []
|
||||
for param_json in json_data.get("arguments", []) + json_data.get("inputs", []): # TODO: deprecate "arguments" keyword
|
||||
param_json["mode"] = "r"
|
||||
self._inputs.append(RemoteProperty(param_json, parent))
|
||||
|
||||
self._outputs = []
|
||||
for param_json in json_data.get("outputs", []): # TODO: deprecate "arguments" keyword
|
||||
param_json["mode"] = "r"
|
||||
self._outputs.append(RemoteProperty(param_json, parent))
|
||||
|
||||
def __call__(self, *args):
|
||||
if (len(self._inputs) != len(args)):
|
||||
raise TypeError("expected {} arguments but have {}".format(len(self._inputs), len(args)))
|
||||
for i in range(len(args)):
|
||||
self._inputs[i].set_value(args[i])
|
||||
self._parent.__channel__.remote_endpoint_operation(self._trigger_id, None, True, 0)
|
||||
if len(self._outputs) > 0:
|
||||
return self._outputs[0].get_value()
|
||||
|
||||
def _dump(self):
|
||||
return "{}({})".format(self._name, ", ".join("{}: {}".format(x._name, x._property_type.__name__) for x in self._inputs))
|
||||
|
||||
class RemoteObject(object):
|
||||
"""
|
||||
Object with functions and properties that map to remote endpoints
|
||||
"""
|
||||
def __init__(self, json_data, parent, channel, logger):
|
||||
"""
|
||||
Creates an object that implements the specified JSON type description by
|
||||
communicating over the provided channel
|
||||
"""
|
||||
# Directly write to __dict__ to avoid calling __setattr__ too early
|
||||
object.__getattribute__(self, "__dict__")["_remote_attributes"] = {}
|
||||
object.__getattribute__(self, "__dict__")["__sealed__"] = False
|
||||
# Assign once more to make linter happy
|
||||
self._remote_attributes = {}
|
||||
self.__sealed__ = False
|
||||
|
||||
self.__channel__ = channel
|
||||
self.__parent__ = parent
|
||||
|
||||
# Build attribute list from JSON
|
||||
for member_json in json_data.get("members", []):
|
||||
member_name = member_json.get("name", None)
|
||||
if member_name is None:
|
||||
logger.debug("ignoring unnamed attribute")
|
||||
continue
|
||||
|
||||
try:
|
||||
type_str = member_json.get("type", None)
|
||||
if type_str == "object":
|
||||
attribute = RemoteObject(member_json, self, channel, logger)
|
||||
elif type_str == "function":
|
||||
attribute = RemoteFunction(member_json, self)
|
||||
elif type_str != None:
|
||||
attribute = RemoteProperty(member_json, self)
|
||||
else:
|
||||
raise ObjectDefinitionError("no type information")
|
||||
except ObjectDefinitionError as ex:
|
||||
logger.debug("malformed member {}: {}".format(member_name, str(ex)))
|
||||
continue
|
||||
|
||||
self._remote_attributes[member_name] = attribute
|
||||
self.__dict__[member_name] = attribute
|
||||
|
||||
# Ensure that from here on out assignments to undefined attributes
|
||||
# raise an exception
|
||||
self.__sealed__ = True
|
||||
channel._channel_broken.subscribe(self._tear_down)
|
||||
|
||||
def _dump(self, indent, depth):
|
||||
if depth <= 0:
|
||||
return "..."
|
||||
lines = []
|
||||
for key, val in self._remote_attributes.items():
|
||||
if isinstance(val, RemoteObject):
|
||||
val_str = indent + key + (": " if depth == 1 else ":\n") + val._dump(indent + " ", depth - 1)
|
||||
else:
|
||||
val_str = indent + val._dump()
|
||||
lines.append(val_str)
|
||||
return "\n".join(lines)
|
||||
|
||||
def __str__(self):
|
||||
return self._dump("", depth=2)
|
||||
|
||||
def __repr__(self):
|
||||
return self.__str__()
|
||||
|
||||
def __getattribute__(self, name):
|
||||
attr = object.__getattribute__(self, "_remote_attributes").get(name, None)
|
||||
if isinstance(attr, RemoteProperty):
|
||||
if attr._can_read:
|
||||
return attr.get_value()
|
||||
else:
|
||||
raise Exception("Cannot read from property {}".format(name))
|
||||
elif attr != None:
|
||||
return attr
|
||||
else:
|
||||
return object.__getattribute__(self, name)
|
||||
#raise AttributeError("Attribute {} not found".format(name))
|
||||
|
||||
def __setattr__(self, name, value):
|
||||
attr = object.__getattribute__(self, "_remote_attributes").get(name, None)
|
||||
if isinstance(attr, RemoteProperty):
|
||||
if attr._can_write:
|
||||
attr.set_value(value)
|
||||
else:
|
||||
raise Exception("Cannot write to property {}".format(name))
|
||||
elif not object.__getattribute__(self, "__sealed__") or name in object.__getattribute__(self, "__dict__"):
|
||||
object.__getattribute__(self, "__dict__")[name] = value
|
||||
else:
|
||||
raise AttributeError("Attribute {} not found".format(name))
|
||||
|
||||
def _tear_down(self):
|
||||
# Clear all remote members
|
||||
for k in self._remote_attributes.keys():
|
||||
self.__dict__.pop(k)
|
||||
self._remote_attributes = {}
|
||||
@@ -0,0 +1,108 @@
|
||||
"""
|
||||
Provides classes that implement the StreamSource/StreamSink and
|
||||
PacketSource/PacketSink interfaces for serial ports.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
import traceback
|
||||
import serial
|
||||
import serial.tools.list_ports
|
||||
import fibre
|
||||
from fibre.utils import TimeoutError
|
||||
|
||||
# TODO: make this customizable
|
||||
DEFAULT_BAUDRATE = 115200
|
||||
|
||||
class SerialStreamTransport(fibre.protocol.StreamSource, fibre.protocol.StreamSink):
|
||||
def __init__(self, port, baud):
|
||||
self._timeout = 1
|
||||
self._dev = serial.Serial(port, baud, timeout=self._timeout)
|
||||
|
||||
def process_bytes(self, bytes):
|
||||
self._dev.write(bytes)
|
||||
|
||||
def get_bytes(self, n_bytes, deadline):
|
||||
"""
|
||||
Returns n bytes unless the deadline is reached, in which case the bytes
|
||||
that were read up to that point are returned. If deadline is None the
|
||||
function blocks forever. A deadline before the current time corresponds
|
||||
to non-blocking mode.
|
||||
"""
|
||||
# Only set new timeout value if it is reasonably different from the old one (e.g. 20% as below)
|
||||
# Otherwise it adds significant overhead (at least under Win10) as the port is reset with every reconfiguration
|
||||
if deadline is None and self._timeout is not None:
|
||||
self._timeout = None
|
||||
self._dev.timeout = None
|
||||
elif deadline is not None:
|
||||
new_timeout = max(deadline - time.monotonic(), 0)
|
||||
if abs(new_timeout - self._timeout) > self._timeout * 0.2:
|
||||
self._timeout = new_timeout
|
||||
self._dev.timeout = new_timeout
|
||||
return self._dev.read(n_bytes)
|
||||
|
||||
def get_bytes_or_fail(self, n_bytes, deadline):
|
||||
result = self.get_bytes(n_bytes, deadline)
|
||||
if len(result) < n_bytes:
|
||||
raise TimeoutError("expected {} bytes but got only {}", n_bytes, len(result))
|
||||
return result
|
||||
|
||||
def close(self):
|
||||
self._dev.close()
|
||||
|
||||
|
||||
def find_dev_serial_ports():
|
||||
try:
|
||||
return ['/dev/' + x for x in os.listdir('/dev')]
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
|
||||
def find_pyserial_ports():
|
||||
return [x.device for x in serial.tools.list_ports.comports()]
|
||||
|
||||
|
||||
def discover_channels(path, serial_number, callback, cancellation_token, channel_termination_token, logger):
|
||||
"""
|
||||
Scans for serial ports that match the path spec.
|
||||
This function blocks until cancellation_token is set.
|
||||
Channels spawned by this function run until channel_termination_token is set.
|
||||
"""
|
||||
if path == None:
|
||||
# This regex should match all desired port names on macOS,
|
||||
# Linux and Windows but might match some incorrect port names.
|
||||
regex = r'^(/dev/tty\.usbmodem.*|/dev/ttyACM.*|COM[0-9]+)$'
|
||||
else:
|
||||
regex = "^" + path + "$"
|
||||
|
||||
known_devices = []
|
||||
def device_matcher(port_name):
|
||||
if port_name in known_devices:
|
||||
return False
|
||||
return bool(re.match(regex, port_name))
|
||||
|
||||
def did_disconnect(port_name, device):
|
||||
device.close()
|
||||
# TODO: yes there is a race condition here in case you wonder.
|
||||
known_devices.pop(known_devices.index(port_name))
|
||||
|
||||
while not cancellation_token.is_set():
|
||||
all_ports = find_pyserial_ports() + find_dev_serial_ports()
|
||||
new_ports = filter(device_matcher, all_ports)
|
||||
for port_name in new_ports:
|
||||
try:
|
||||
serial_device = SerialStreamTransport(port_name, DEFAULT_BAUDRATE)
|
||||
input_stream = fibre.protocol.PacketFromStreamConverter(serial_device)
|
||||
output_stream = fibre.protocol.StreamBasedPacketSink(serial_device)
|
||||
channel = fibre.protocol.Channel(
|
||||
"serial port {}@{}".format(port_name, DEFAULT_BAUDRATE),
|
||||
input_stream, output_stream, channel_termination_token, logger)
|
||||
channel.serial_device = serial_device
|
||||
except serial.serialutil.SerialException:
|
||||
logger.debug("Serial device init failed. Ignoring this port. More info: " + traceback.format_exc())
|
||||
known_devices.append(port_name)
|
||||
else:
|
||||
known_devices.append(port_name)
|
||||
channel._channel_broken.subscribe(lambda: did_disconnect(port_name, serial_device))
|
||||
callback(channel)
|
||||
time.sleep(1)
|
||||
@@ -0,0 +1,128 @@
|
||||
|
||||
import sys
|
||||
import platform
|
||||
import threading
|
||||
import fibre
|
||||
|
||||
def did_discover_device(device,
|
||||
interactive_variables, discovered_devices,
|
||||
branding_short, branding_long,
|
||||
logger, app_shutdown_token):
|
||||
"""
|
||||
Handles the discovery of new devices by displaying a
|
||||
message and making the device available to the interactive
|
||||
console
|
||||
"""
|
||||
serial_number = '{:012X}'.format(device.serial_number) if hasattr(device, 'serial_number') else "[unknown serial number]"
|
||||
if serial_number in discovered_devices:
|
||||
verb = "Reconnected"
|
||||
index = discovered_devices.index(serial_number)
|
||||
else:
|
||||
verb = "Connected"
|
||||
discovered_devices.append(serial_number)
|
||||
index = len(discovered_devices) - 1
|
||||
interactive_name = branding_short + str(index)
|
||||
|
||||
# Publish new device to interactive console
|
||||
interactive_variables[interactive_name] = device
|
||||
globals()[interactive_name] = device # Add to globals so tab complete works
|
||||
logger.notify("{} to {} {} as {}".format(verb, branding_long, serial_number, interactive_name))
|
||||
|
||||
# Subscribe to disappearance of the device
|
||||
device.__channel__._channel_broken.subscribe(lambda: did_lose_device(interactive_name, logger, app_shutdown_token))
|
||||
|
||||
def did_lose_device(interactive_name, logger, app_shutdown_token):
|
||||
"""
|
||||
Handles the disappearance of a device by displaying
|
||||
a message.
|
||||
"""
|
||||
if not app_shutdown_token.is_set():
|
||||
logger.warn("Oh no {} disappeared".format(interactive_name))
|
||||
|
||||
def launch_shell(args,
|
||||
interactive_variables,
|
||||
print_banner, print_help,
|
||||
logger, app_shutdown_token,
|
||||
branding_short="dev", branding_long="device"):
|
||||
"""
|
||||
Launches an interactive python or IPython command line
|
||||
interface.
|
||||
As devices are connected they are made available as
|
||||
"dev0", "dev1", ...
|
||||
The names of the variables can be customized by setting branding_short.
|
||||
"""
|
||||
|
||||
discovered_devices = []
|
||||
globals().update(interactive_variables)
|
||||
|
||||
# Connect to device
|
||||
logger.debug("Waiting for {}...".format(branding_long))
|
||||
fibre.find_all(args.path, args.serial_number,
|
||||
lambda dev: did_discover_device(dev, interactive_variables, discovered_devices, branding_short, branding_long, logger, app_shutdown_token),
|
||||
app_shutdown_token,
|
||||
app_shutdown_token,
|
||||
logger=logger)
|
||||
|
||||
# Check if IPython is installed
|
||||
if args.no_ipython:
|
||||
use_ipython = False
|
||||
else:
|
||||
try:
|
||||
import IPython
|
||||
use_ipython = True
|
||||
except:
|
||||
print("Warning: you don't have IPython installed.")
|
||||
print("If you want to have an improved interactive console with pretty colors,")
|
||||
print("you should install IPython\n")
|
||||
use_ipython = False
|
||||
|
||||
interactive_variables["help"] = lambda: print_help(args, len(discovered_devices) > 0)
|
||||
|
||||
# If IPython is installed, embed IPython shell, otherwise embed regular shell
|
||||
if use_ipython:
|
||||
# Override help function # pylint: disable=W0612
|
||||
help = lambda: print_help(args, len(discovered_devices) > 0)
|
||||
# to fix broken "%run -i script.py"
|
||||
locals()['__name__'] = globals()['__name__']
|
||||
console = IPython.terminal.embed.InteractiveShellEmbed(banner1='')
|
||||
|
||||
# hack to make IPython look like the regular console
|
||||
console.runcode = console.run_cell
|
||||
interact = console
|
||||
|
||||
# Catch ChannelBrokenException (since disconnect is not always an error)
|
||||
default_exception_hook = console._showtraceback
|
||||
def filtered_exception_hook(ex_class, ex, trace):
|
||||
if(ex_class.__module__+'.'+ex_class.__name__ != 'fibre.protocol.ChannelBrokenException'):
|
||||
default_exception_hook(ex_class,ex,trace)
|
||||
|
||||
console._showtraceback = filtered_exception_hook
|
||||
else:
|
||||
# Enable tab complete if possible
|
||||
try:
|
||||
import readline # Works only on Unix
|
||||
readline.parse_and_bind("tab: complete")
|
||||
except:
|
||||
sudo_prefix = "" if platform.system() == "Windows" else "sudo "
|
||||
print("Warning: could not enable tab-complete. User experience will suffer.\n"
|
||||
"Run `{}pip install readline` and then restart this script to fix this."
|
||||
.format(sudo_prefix))
|
||||
|
||||
import code
|
||||
console = code.InteractiveConsole(locals=interactive_variables)
|
||||
interact = lambda: console.interact(banner='')
|
||||
|
||||
# Catch ChannelBrokenException (since disconnect is not alway an error)
|
||||
console.runcode("import sys")
|
||||
console.runcode("default_exception_hook = sys.excepthook")
|
||||
console.runcode("def filtered_exception_hook(ex_class, ex, trace):\n"
|
||||
" if ex_class.__module__ + '.' + ex_class.__name__ != 'fibre.protocol.ChannelBrokenException':\n"
|
||||
" default_exception_hook(ex_class,ex,trace)")
|
||||
console.runcode("sys.excepthook=filtered_exception_hook")
|
||||
|
||||
|
||||
# Launch shell
|
||||
print_banner()
|
||||
logger._skip_bottom_line = True
|
||||
interact()
|
||||
app_shutdown_token.set()
|
||||
@@ -0,0 +1,85 @@
|
||||
|
||||
import sys
|
||||
import socket
|
||||
import time
|
||||
import traceback
|
||||
import fibre.protocol
|
||||
from fibre.utils import wait_any, TimeoutError
|
||||
|
||||
def noprint(x):
|
||||
pass
|
||||
|
||||
class TCPTransport(fibre.protocol.StreamSource, fibre.protocol.StreamSink):
|
||||
def __init__(self, dest_addr, dest_port, logger):
|
||||
# TODO: FIXME: use IPv6
|
||||
# Problem: getaddrinfo fails if the resolver returns an
|
||||
# IPv4 address, but we are using AF_INET6
|
||||
#family = socket.AF_INET6 if socket.has_ipv6 else socket.AF_INET
|
||||
family = socket.AF_INET
|
||||
self.sock = socket.socket(family, socket.SOCK_STREAM)
|
||||
# TODO: Determine the right address to use from the list
|
||||
self.target = socket.getaddrinfo(dest_addr, dest_port, family)[0][4]
|
||||
# TODO: this blocks until a connection is established, or the system cancels it
|
||||
self.sock.connect(self.target)
|
||||
|
||||
def process_bytes(self, buffer):
|
||||
self.sock.send(buffer)
|
||||
|
||||
def get_bytes(self, n_bytes, deadline):
|
||||
"""
|
||||
Returns n bytes unless the deadline is reached, in which case the bytes
|
||||
that were read up to that point are returned. If deadline is None the
|
||||
function blocks forever. A deadline before the current time corresponds
|
||||
to non-blocking mode.
|
||||
"""
|
||||
# convert deadline to seconds (floating point)
|
||||
deadline = None if deadline is None else max(deadline - time.monotonic(), 0)
|
||||
self.sock.settimeout(deadline)
|
||||
try:
|
||||
data = self.sock.recv(n_bytes, socket.MSG_WAITALL) # receive n_bytes
|
||||
return data
|
||||
except socket.timeout:
|
||||
# if we got a timeout data will still be none, so we call recv again
|
||||
# this time in non blocking state and see if we can get some data
|
||||
try:
|
||||
return self.sock.recv(n_bytes, socket.MSG_DONTWAIT)
|
||||
except socket.timeout:
|
||||
raise TimeoutError
|
||||
|
||||
def get_bytes_or_fail(self, n_bytes, deadline):
|
||||
result = self.get_bytes(n_bytes, deadline)
|
||||
if len(result) < n_bytes:
|
||||
raise TimeoutError("expected {} bytes but got only {}".format(n_bytes, len(result)))
|
||||
return result
|
||||
|
||||
|
||||
|
||||
def discover_channels(path, serial_number, callback, cancellation_token, channel_termination_token, logger):
|
||||
"""
|
||||
Tries to connect to a TCP server based on the path spec.
|
||||
This function blocks until cancellation_token is set.
|
||||
Channels spawned by this function run until channel_termination_token is set.
|
||||
"""
|
||||
try:
|
||||
dest_addr = ':'.join(path.split(":")[:-1])
|
||||
dest_port = int(path.split(":")[-1])
|
||||
except (ValueError, IndexError):
|
||||
raise Exception('"{}" is not a valid TCP destination. The format should be something like "localhost:1234".'
|
||||
.format(path))
|
||||
|
||||
while not cancellation_token.is_set():
|
||||
try:
|
||||
tcp_transport = fibre.tcp_transport.TCPTransport(dest_addr, dest_port, logger)
|
||||
stream2packet_input = fibre.protocol.PacketFromStreamConverter(tcp_transport)
|
||||
packet2stream_output = fibre.protocol.StreamBasedPacketSink(tcp_transport)
|
||||
channel = fibre.protocol.Channel(
|
||||
"TCP device {}:{}".format(dest_addr, dest_port),
|
||||
stream2packet_input, packet2stream_output,
|
||||
channel_termination_token, logger)
|
||||
except:
|
||||
#logger.debug("TCP channel init failed. More info: " + traceback.format_exc())
|
||||
pass
|
||||
else:
|
||||
callback(channel)
|
||||
wait_any(None, cancellation_token, channel._channel_broken)
|
||||
time.sleep(1)
|
||||
@@ -0,0 +1,57 @@
|
||||
|
||||
import sys
|
||||
import socket
|
||||
import time
|
||||
import traceback
|
||||
import fibre.protocol
|
||||
from fibre.utils import wait_any
|
||||
|
||||
def noprint(x):
|
||||
pass
|
||||
|
||||
class UDPTransport(fibre.protocol.PacketSource, fibre.protocol.PacketSink):
|
||||
def __init__(self, dest_addr, dest_port, logger):
|
||||
# TODO: FIXME: use IPv6
|
||||
# Problem: getaddrinfo fails if the resolver returns an
|
||||
# IPv4 address, but we are using AF_INET6
|
||||
#family = socket.AF_INET6 if socket.has_ipv6 else socket.AF_INET
|
||||
family = socket.AF_INET
|
||||
self.sock = socket.socket(family, socket.SOCK_DGRAM)
|
||||
# TODO: Determine the right address to use from the list
|
||||
self.target = socket.getaddrinfo(dest_addr,dest_port, family)[0][4]
|
||||
|
||||
def process_packet(self, buffer):
|
||||
self.sock.sendto(buffer, self.target)
|
||||
|
||||
def get_packet(self, deadline):
|
||||
# TODO: implement deadline
|
||||
data, _ = self.sock.recvfrom(1024)
|
||||
return data
|
||||
|
||||
def discover_channels(path, serial_number, callback, cancellation_token, channel_termination_token, logger):
|
||||
"""
|
||||
Tries to connect to a UDP server based on the path spec.
|
||||
This function blocks until cancellation_token is set.
|
||||
Channels spawned by this function run until channel_termination_token is set.
|
||||
"""
|
||||
try:
|
||||
dest_addr = ':'.join(path.split(":")[:-1])
|
||||
dest_port = int(path.split(":")[-1])
|
||||
except (ValueError, IndexError):
|
||||
raise Exception('"{}" is not a valid UDP destination. The format should be something like "localhost:1234".'
|
||||
.format(path))
|
||||
|
||||
while not cancellation_token.is_set():
|
||||
try:
|
||||
udp_transport = fibre.udp_transport.UDPTransport(dest_addr, dest_port, logger)
|
||||
channel = fibre.protocol.Channel(
|
||||
"UDP device {}:{}".format(dest_addr, dest_port),
|
||||
udp_transport, udp_transport,
|
||||
channel_termination_token, logger)
|
||||
except:
|
||||
logger.debug("UDP channel init failed. More info: " + traceback.format_exc())
|
||||
pass
|
||||
else:
|
||||
callback(channel)
|
||||
wait_any(None, cancellation_token, channel._channel_broken)
|
||||
time.sleep(1)
|
||||
@@ -0,0 +1,216 @@
|
||||
# requires pyusb
|
||||
# pip install --pre pyusb
|
||||
|
||||
import usb.core
|
||||
import usb.util
|
||||
import sys
|
||||
import time
|
||||
import fibre.protocol
|
||||
import traceback
|
||||
import platform
|
||||
from fibre.utils import TimeoutError
|
||||
|
||||
# Currently we identify fibre-enabled devices by VID,PID
|
||||
# TODO: identify by USB descriptors
|
||||
WELL_KNOWN_VID_PID_PAIRS = [
|
||||
(0x1209, 0x0D31),
|
||||
(0x1209, 0x0D32),
|
||||
(0x1209, 0x0D33)
|
||||
]
|
||||
|
||||
class USBBulkTransport(fibre.protocol.PacketSource, fibre.protocol.PacketSink):
|
||||
def __init__(self, dev, logger):
|
||||
self._logger = logger
|
||||
self.dev = dev
|
||||
self.intf = None
|
||||
self._name = "USB device {}:{}".format(dev.idVendor, dev.idProduct)
|
||||
self._was_damaged = False
|
||||
|
||||
##
|
||||
# information about the connected device
|
||||
##
|
||||
def info(self):
|
||||
# loop through configurations
|
||||
string = ""
|
||||
for cfg in self.dev:
|
||||
string += "ConfigurationValue {0}\n".format(cfg.bConfigurationValue)
|
||||
for intf in cfg:
|
||||
string += "\tInterfaceNumber {0},{1}\n".format(intf.bInterfaceNumber, intf.bAlternateSetting)
|
||||
for ep in intf:
|
||||
string += "\t\tEndpointAddress {0}\n".format(ep.bEndpointAddress)
|
||||
return string
|
||||
|
||||
def init(self):
|
||||
# Under some conditions, the Linux USB/libusb stack ends up in a corrupt
|
||||
# state where there are a few packets in a receive queue but a call
|
||||
# to epr.read() does not return these packet until a new packet arrives.
|
||||
# This undesirable queue can be cleared by resetting the device.
|
||||
# On windows this would cause file-not-found errors in subsequent dev calls
|
||||
if platform.system() != 'Windows':
|
||||
self.dev.reset()
|
||||
|
||||
#self.dev.set_configuration() # no args: set first configuration
|
||||
|
||||
# Find the best interface
|
||||
self.cfg = self.dev.get_active_configuration()
|
||||
custom_interfaces = [i for i in self.cfg.interfaces() if i.bInterfaceClass == 0x00 and i.bInterfaceSubClass == 0x01]
|
||||
cdc_interfaces = [i for i in self.cfg.interfaces() if i.bInterfaceClass == 0x0a and i.bInterfaceSubClass == 0x00]
|
||||
all_compatible_interfaces = custom_interfaces + cdc_interfaces
|
||||
if len(all_compatible_interfaces) == 0:
|
||||
raise Exception("the device has no compatible interfaces")
|
||||
self.intf = all_compatible_interfaces[0]
|
||||
|
||||
# Try to detach kernel driver from interface
|
||||
try:
|
||||
if self.dev.is_kernel_driver_active(self.intf.bInterfaceNumber):
|
||||
self.dev.detach_kernel_driver(self.intf.bInterfaceNumber)
|
||||
self._logger.debug("Detached Kernel Driver")
|
||||
else:
|
||||
self._logger.debug("Kernel Driver was not attached")
|
||||
except NotImplementedError:
|
||||
pass #is_kernel_driver_active not implemented on Windows
|
||||
|
||||
# find write endpoint (first OUT endpoint)
|
||||
self.epw = usb.util.find_descriptor(self.intf,
|
||||
custom_match = \
|
||||
lambda e: \
|
||||
usb.util.endpoint_direction(e.bEndpointAddress) == \
|
||||
usb.util.ENDPOINT_OUT
|
||||
)
|
||||
assert self.epw is not None
|
||||
self._logger.debug("EndpointAddress for writing {}".format(self.epw.bEndpointAddress))
|
||||
# find read endpoint (first IN endpoint)
|
||||
self.epr = usb.util.find_descriptor(self.intf,
|
||||
custom_match = \
|
||||
lambda e: \
|
||||
usb.util.endpoint_direction(e.bEndpointAddress) == \
|
||||
usb.util.ENDPOINT_IN
|
||||
)
|
||||
assert self.epr is not None
|
||||
self._logger.debug("EndpointAddress for reading {}".format(self.epr.bEndpointAddress))
|
||||
|
||||
def deinit(self):
|
||||
if not self.intf is None:
|
||||
usb.util.release_interface(self.dev, self.intf)
|
||||
|
||||
def process_packet(self, usbBuffer):
|
||||
try:
|
||||
ret = self.epw.write(usbBuffer, 0)
|
||||
if self._was_damaged:
|
||||
self._logger.debug("Recovered from USB halt/stall condition")
|
||||
self._was_damaged = False
|
||||
return ret
|
||||
except usb.core.USBError as ex:
|
||||
if ex.errno == 19 or ex.errno == 32: # "no such device", "pipe error"
|
||||
raise fibre.protocol.ChannelBrokenException()
|
||||
elif ex.errno is None or ex.errno == 60 or ex.errno == 110: # timeout
|
||||
raise TimeoutError()
|
||||
else:
|
||||
self._logger.debug("error in usbbulk_transport.py, process_packet")
|
||||
self._logger.debug(traceback.format_exc())
|
||||
self._logger.debug("halt condition: {}".format(ex.errno))
|
||||
self._logger.debug(str(ex))
|
||||
# Try resetting halt/stall condition
|
||||
try:
|
||||
self.deinit()
|
||||
self.init()
|
||||
except usb.core.USBError:
|
||||
raise fibre.protocol.ChannelBrokenException()
|
||||
# Retry transfer
|
||||
self._was_damaged = True
|
||||
raise fibre.protocol.ChannelDamagedException()
|
||||
|
||||
def get_packet(self, deadline):
|
||||
try:
|
||||
bufferLen = self.epr.wMaxPacketSize
|
||||
timeout = max(int((deadline - time.monotonic()) * 1000), 0)
|
||||
ret = self.epr.read(bufferLen, timeout)
|
||||
if self._was_damaged:
|
||||
self._logger.debug("Recovered from USB halt/stall condition")
|
||||
self._was_damaged = False
|
||||
return bytearray(ret)
|
||||
except usb.core.USBError as ex:
|
||||
if ex.errno == 19 or ex.errno == 32: # "no such device", "pipe error"
|
||||
raise fibre.protocol.ChannelBrokenException()
|
||||
elif ex.errno is None or ex.errno == 60 or ex.errno == 110: # timeout
|
||||
raise TimeoutError()
|
||||
else:
|
||||
self._logger.debug("error in usbbulk_transport.py, process_packet")
|
||||
self._logger.debug(traceback.format_exc())
|
||||
self._logger.debug("halt condition: {}".format(ex.errno))
|
||||
self._logger.debug(str(ex))
|
||||
# Try resetting halt/stall condition
|
||||
try:
|
||||
self.deinit()
|
||||
self.init()
|
||||
except usb.core.USBError:
|
||||
raise fibre.protocol.ChannelBrokenException()
|
||||
# Retry transfer
|
||||
self._was_damaged = True
|
||||
raise fibre.protocol.ChannelDamagedException()
|
||||
|
||||
|
||||
def discover_channels(path, serial_number, callback, cancellation_token, channel_termination_token, logger):
|
||||
"""
|
||||
Scans for USB devices that match the path spec.
|
||||
This function blocks until cancellation_token is set.
|
||||
Channels spawned by this function run until channel_termination_token is set.
|
||||
"""
|
||||
if path == None or path == "":
|
||||
bus = None
|
||||
address = None
|
||||
else:
|
||||
try:
|
||||
bus = int(path.split(":")[0])
|
||||
address = int(path.split(":")[1])
|
||||
except (ValueError, IndexError):
|
||||
raise Exception("{} is not a valid USB path specification. "
|
||||
"Expected a string of the format BUS:DEVICE where BUS "
|
||||
"and DEVICE are integers.".format(path))
|
||||
|
||||
known_devices = []
|
||||
def device_matcher(device):
|
||||
#print(" test {:04X}:{:04X}".format(device.idVendor, device.idProduct))
|
||||
try:
|
||||
if (device.bus, device.address) in known_devices:
|
||||
return False
|
||||
if bus != None and device.bus != bus:
|
||||
return False
|
||||
if address != None and device.address != address:
|
||||
return False
|
||||
if serial_number != None and device.serial_number != serial_number:
|
||||
return False
|
||||
if (device.idVendor, device.idProduct) not in WELL_KNOWN_VID_PID_PAIRS:
|
||||
return False
|
||||
except:
|
||||
return False
|
||||
return True
|
||||
|
||||
while not cancellation_token.is_set():
|
||||
# logger.debug("USB discover loop")
|
||||
devices = usb.core.find(find_all=True, custom_match=device_matcher)
|
||||
for usb_device in devices:
|
||||
try:
|
||||
bulk_device = USBBulkTransport(usb_device, logger)
|
||||
logger.debug(bulk_device.info())
|
||||
bulk_device.init()
|
||||
channel = fibre.protocol.Channel(
|
||||
"USB device bus {} device {}".format(usb_device.bus, usb_device.address),
|
||||
bulk_device, bulk_device, channel_termination_token, logger)
|
||||
channel.usb_device = usb_device # for debugging only
|
||||
except usb.core.USBError as ex:
|
||||
if ex.errno == 13:
|
||||
# TODO: this is an ODrive specific message and should live outside of the fibre library
|
||||
logger.warn("I found a USB device that looks like an ODrive (bus {}, device {}) but I can't access it. Try running `sudo odrivetool udev-setup`, then unplug and replug the device.".format(usb_device.bus, usb_device.address))
|
||||
known_devices.append((usb_device.bus, usb_device.address))
|
||||
elif ex.errno == 16:
|
||||
logger.debug("USB device busy. I'll reset it and try again.")
|
||||
usb_device.reset()
|
||||
continue
|
||||
else:
|
||||
logger.warn("USB device init failed (bus {}, device {}). Ignoring this device. More info: ".format(usb_device.bus, usb_device.address) + traceback.format_exc())
|
||||
known_devices.append((usb_device.bus, usb_device.address))
|
||||
else:
|
||||
known_devices.append((usb_device.bus, usb_device.address))
|
||||
callback(channel)
|
||||
time.sleep(1)
|
||||
@@ -0,0 +1,237 @@
|
||||
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
import platform
|
||||
import subprocess
|
||||
import os
|
||||
|
||||
try:
|
||||
if platform.system() == 'Windows':
|
||||
import win32console
|
||||
# TODO: we should win32console anyway so we could just omit colorama
|
||||
import colorama
|
||||
colorama.init()
|
||||
except ImportError:
|
||||
print("Could not init terminal features.")
|
||||
sys.stdout.flush()
|
||||
pass
|
||||
|
||||
if sys.version_info < (3, 3):
|
||||
class TimeoutError(Exception):
|
||||
pass
|
||||
else:
|
||||
TimeoutError = TimeoutError
|
||||
|
||||
def get_serial_number_str(device):
|
||||
if hasattr(device, 'serial_number'):
|
||||
return format(device.serial_number, 'x').upper()
|
||||
else:
|
||||
return "[unknown serial number]"
|
||||
|
||||
## Threading utils ##
|
||||
class Event():
|
||||
"""
|
||||
Alternative to threading.Event(), enhanced by the subscribe() function
|
||||
that the original fails to provide.
|
||||
@param Trigger: if supplied, the newly created event will be triggered
|
||||
as soon as the trigger event becomes set
|
||||
"""
|
||||
def __init__(self, trigger=None):
|
||||
self._evt = threading.Event()
|
||||
self._subscribers = []
|
||||
self._mutex = threading.Lock()
|
||||
if not trigger is None:
|
||||
trigger.subscribe(lambda: self.set())
|
||||
|
||||
def is_set(self):
|
||||
return self._evt.is_set()
|
||||
|
||||
def set(self):
|
||||
"""
|
||||
Sets the event and invokes all subscribers if the event was
|
||||
not already set
|
||||
"""
|
||||
self._mutex.acquire()
|
||||
try:
|
||||
if not self._evt.is_set():
|
||||
self._evt.set()
|
||||
for s in self._subscribers:
|
||||
s()
|
||||
finally:
|
||||
self._mutex.release()
|
||||
|
||||
def subscribe(self, handler):
|
||||
"""
|
||||
Invokes the specified handler exactly once as soon as the
|
||||
specified event is set. If the event is already set, the
|
||||
handler is invoked immediately.
|
||||
Returns a function that can be invoked to unsubscribe.
|
||||
"""
|
||||
if handler is None:
|
||||
raise TypeError
|
||||
self._mutex.acquire()
|
||||
try:
|
||||
self._subscribers.append(handler)
|
||||
if self._evt.is_set():
|
||||
handler()
|
||||
finally:
|
||||
self._mutex.release()
|
||||
return handler
|
||||
|
||||
def unsubscribe(self, handler):
|
||||
self._mutex.acquire()
|
||||
try:
|
||||
self._subscribers.pop(self._subscribers.index(handler))
|
||||
finally:
|
||||
self._mutex.release()
|
||||
|
||||
def wait(self, timeout=None):
|
||||
if not self._evt.wait(timeout=timeout):
|
||||
raise TimeoutError()
|
||||
|
||||
def trigger_after(self, timeout):
|
||||
"""
|
||||
Triggers the event after the specified timeout.
|
||||
This function returns immediately.
|
||||
"""
|
||||
def delayed_trigger():
|
||||
if not self.wait(timeout=timeout):
|
||||
self.set()
|
||||
threading.Thread(target=delayed_trigger)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
|
||||
def wait_any(timeout=None, *events):
|
||||
"""
|
||||
Blocks until any of the specified events are triggered.
|
||||
Returns the index of the event that was triggerd or raises
|
||||
a TimeoutError
|
||||
Param timeout: A timeout in seconds
|
||||
"""
|
||||
or_event = threading.Event()
|
||||
subscriptions = []
|
||||
for event in events:
|
||||
subscriptions.append((event, event.subscribe(lambda: or_event.set())))
|
||||
or_event.wait(timeout=timeout)
|
||||
for event, sub in subscriptions:
|
||||
event.unsubscribe(sub)
|
||||
for i in range(len(events)):
|
||||
if events[i].is_set():
|
||||
return i
|
||||
raise TimeoutError()
|
||||
|
||||
|
||||
## Log utils ##
|
||||
|
||||
class Logger():
|
||||
"""
|
||||
Logs messages to stdout
|
||||
"""
|
||||
|
||||
COLOR_DEFAULT = 0
|
||||
COLOR_GREEN = 1
|
||||
COLOR_CYAN = 2
|
||||
COLOR_YELLOW = 3
|
||||
COLOR_RED = 4
|
||||
|
||||
_VT100Colors = {
|
||||
COLOR_GREEN: '\x1b[92;1m',
|
||||
COLOR_CYAN: '\x1b[96;1m',
|
||||
COLOR_YELLOW: '\x1b[93;1m',
|
||||
COLOR_RED: '\x1b[91;1m',
|
||||
COLOR_DEFAULT: '\x1b[0m'
|
||||
}
|
||||
|
||||
_Win32Colors = {
|
||||
COLOR_GREEN: 0x0A,
|
||||
COLOR_CYAN: 0x0B,
|
||||
COLOR_YELLOW: 0x0E,
|
||||
COLOR_RED: 0x0C,
|
||||
COLOR_DEFAULT: 0x07
|
||||
}
|
||||
|
||||
def __init__(self, verbose=True):
|
||||
self._prefix = ''
|
||||
self._skip_bottom_line = False # If true, messages are printed one line above the cursor
|
||||
self._verbose = verbose
|
||||
self._print_lock = threading.Lock()
|
||||
if platform.system() == 'Windows':
|
||||
self._stdout_buf = win32console.GetStdHandle(win32console.STD_OUTPUT_HANDLE)
|
||||
|
||||
def indent(self, prefix=' '):
|
||||
indented_logger = Logger()
|
||||
indented_logger._prefix = self._prefix + prefix
|
||||
return indented_logger
|
||||
|
||||
def print_on_second_last_line(self, text, color):
|
||||
"""
|
||||
Prints a text on the second last line.
|
||||
This can be used to print a message above the command
|
||||
prompt. If the command prompt spans multiple lines
|
||||
there will be glitches.
|
||||
If the printed text spans multiple lines there will also
|
||||
be glitches (though this could be fixed).
|
||||
"""
|
||||
|
||||
if platform.system() == 'Windows':
|
||||
# Windows <10 doesn't understand VT100 escape codes and the colorama
|
||||
# also doesn't support the specific escape codes we need so we use the
|
||||
# native Win32 API.
|
||||
info = self._stdout_buf.GetConsoleScreenBufferInfo()
|
||||
cursor_pos = info['CursorPosition']
|
||||
scroll_rect=win32console.PySMALL_RECTType(
|
||||
Left=0, Top=1,
|
||||
Right=info['Window'].Right,
|
||||
Bottom=cursor_pos.Y-1)
|
||||
scroll_dest = win32console.PyCOORDType(scroll_rect.Left, scroll_rect.Top-1)
|
||||
self._stdout_buf.ScrollConsoleScreenBuffer(
|
||||
scroll_rect, scroll_rect, scroll_dest, # clipping rect is same as scroll rect
|
||||
u' ', Logger._Win32Colors[color]) # fill with empty cells with the desired color attributes
|
||||
line_start = win32console.PyCOORDType(0, cursor_pos.Y-1)
|
||||
self._stdout_buf.WriteConsoleOutputCharacter(text, line_start)
|
||||
|
||||
else:
|
||||
# Assume we're in a terminal that interprets VT100 escape codes.
|
||||
# TODO: test on macOS
|
||||
|
||||
# Escape character sequence:
|
||||
# ESC 7: store cursor position
|
||||
# ESC 1A: move cursor up by one
|
||||
# ESC 1S: scroll entire viewport by one
|
||||
# ESC 1L: insert 1 line at cursor position
|
||||
# (print text)
|
||||
# ESC 8: restore old cursor position
|
||||
|
||||
self._print_lock.acquire()
|
||||
sys.stdout.write('\x1b7\x1b[1A\x1b[1S\x1b[1L')
|
||||
sys.stdout.write(Logger._VT100Colors[color] + text + Logger._VT100Colors[Logger.COLOR_DEFAULT])
|
||||
sys.stdout.write('\x1b8')
|
||||
sys.stdout.flush()
|
||||
self._print_lock.release()
|
||||
|
||||
def print_colored(self, text, color):
|
||||
if self._skip_bottom_line:
|
||||
self.print_on_second_last_line(text, color)
|
||||
else:
|
||||
# On Windows, colorama does the job of interpreting the VT100 escape sequences
|
||||
self._print_lock.acquire()
|
||||
sys.stdout.write(Logger._VT100Colors[color] + text + Logger._VT100Colors[Logger.COLOR_DEFAULT] + '\n')
|
||||
sys.stdout.flush()
|
||||
self._print_lock.release()
|
||||
|
||||
def debug(self, text):
|
||||
if self._verbose:
|
||||
self.print_colored(self._prefix + text, Logger.COLOR_DEFAULT)
|
||||
def success(self, text):
|
||||
self.print_colored(self._prefix + text, Logger.COLOR_GREEN)
|
||||
def info(self, text):
|
||||
self.print_colored(self._prefix + text, Logger.COLOR_DEFAULT)
|
||||
def notify(self, text):
|
||||
self.print_colored(self._prefix + text, Logger.COLOR_CYAN)
|
||||
def warn(self, text):
|
||||
self.print_colored(self._prefix + text, Logger.COLOR_YELLOW)
|
||||
def error(self, text):
|
||||
# TODO: write to stderr
|
||||
self.print_colored(self._prefix + text, Logger.COLOR_RED)
|
||||
@@ -0,0 +1,90 @@
|
||||
"""
|
||||
This script is used to deploy the Fibre python library to PyPi
|
||||
so that users can install them easily with
|
||||
"pip install fibre"
|
||||
|
||||
To install the package and its dependencies locally, run:
|
||||
sudo pip install -r requirements.txt
|
||||
|
||||
To build and package the python tools into a tar archive:
|
||||
python setup.py sdist
|
||||
|
||||
Warning: Before you proceed, be aware that you can upload a
|
||||
specific version only once ever. After that you need to increment
|
||||
the hotfix number. Deleting the release manually on the PyPi
|
||||
website does not help.
|
||||
|
||||
Use TestPyPi while developing.
|
||||
|
||||
To build, package and upload the python tools to TestPyPi, run:
|
||||
python setup.py sdist upload -r pypitest
|
||||
To make a real release ensure you're at the release commit
|
||||
and then run the above command without the "test" (so just "pypi").
|
||||
|
||||
To install a prerelease version from test index:
|
||||
sudo pip install --pre --index-url https://test.pypi.org/simple/ --no-cache-dir fibre
|
||||
|
||||
|
||||
PyPi access requires that you have set up ~/.pypirc with your
|
||||
PyPi credentials and that your account has the rights
|
||||
to publish packages with the name fibre.
|
||||
"""
|
||||
|
||||
# TODO: add additional y/n prompt to prevent from erroneous upload
|
||||
|
||||
from setuptools import setup
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Change this if you already uploaded the current
|
||||
# version but need to release a hotfix
|
||||
hotfix = 0
|
||||
|
||||
#creating_package = "sdist" in sys.argv
|
||||
#
|
||||
## Load version from Git tag
|
||||
#import odrive.version
|
||||
#version = odrive.version.get_version_str(git_only=creating_package)
|
||||
#
|
||||
#if creating_package and (hotfix > 0 or not version[-1].isdigit()):
|
||||
# # Add this for hotfixes
|
||||
# version += "-" + str(hotfix)
|
||||
#
|
||||
#
|
||||
## If we're currently creating the package we need to autogenerate
|
||||
## a file that contains the version string
|
||||
#if creating_package:
|
||||
# version_file_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'odrive', 'version.txt')
|
||||
# with open(version_file_path, mode='w') as version_file:
|
||||
# version_file.write(version)
|
||||
#
|
||||
## TODO: find a better place for this
|
||||
#if not creating_package:
|
||||
# import platform
|
||||
# if platform.system() == 'Linux':
|
||||
# import odrive.utils
|
||||
# odrive.utils.setup_udev_rules(odrive.utils.Logger())
|
||||
|
||||
setup(
|
||||
name = 'fibre',
|
||||
packages = ['fibre'],
|
||||
#scripts = ['..fibre', 'odrivetool.bat', 'odrive_demo.py'],
|
||||
version = '0.0.1dev0',
|
||||
description = 'Abstraction layer for painlessly building object oriented distributed systems that just work',
|
||||
author = 'Samuel Sadok',
|
||||
author_email = 'samuel.sadok@bluewin.ch',
|
||||
license='MIT',
|
||||
url = 'https://github.com/samuelsadok/fibre',
|
||||
keywords = ['communication', 'transport-layer', 'rpc'],
|
||||
install_requires = [
|
||||
'appdirs', # Used to find caching directory
|
||||
],
|
||||
#package_data={'': ['version.txt']},
|
||||
classifiers = [],
|
||||
)
|
||||
|
||||
# TODO: include README
|
||||
|
||||
## clean up
|
||||
#if creating_package:
|
||||
# os.remove(version_file_path)
|
||||
@@ -0,0 +1,27 @@
|
||||
|
||||
|
||||
tup.include('../tupfiles/build.lua')
|
||||
tup.include('../cpp/package.lua')
|
||||
|
||||
test_server = define_package{
|
||||
packages={fibre_package},
|
||||
sources={'test_server.cpp'}
|
||||
}
|
||||
|
||||
unit_tests = define_package{
|
||||
packages={fibre_package},
|
||||
sources={'run_tests.cpp'}
|
||||
}
|
||||
|
||||
|
||||
toolchain=GCCToolchain('', 'build', {'-O3', '-fvisibility=hidden', '-frename-registers', '-funroll-loops'}, {})
|
||||
toolchain=GCCToolchain('', 'build', {'-O3', '-g', '-Wall'}, {})
|
||||
--toolchain=GCCToolchain('avr-', {'-Ofast', '-fvisibility=hidden', '-frename-registers', '-funroll-loops', '-I/home/samuel/stlport-avr/stlport'}, {})
|
||||
--toolchain=LLVMToolchain('x86_64', {'-O3', '-fno-sanitize=safe-stack', '-fno-stack-protector'}, {'-flto', '-Wl,-s'})
|
||||
--toolchain=LLVMToolchain('avr', {'-O3', '-std=gnu++11', '--target=avr', '-fno-sanitize=safe-stack', '-fno-stack-protector', '-I/home/samuel/stlport-avr/stlport'}, {'-flto', '-Wl,-s'})
|
||||
|
||||
|
||||
if tup.getconfig("BUILD_FIBRE_TESTS") == "true" then
|
||||
build_executable('test_server', test_server, toolchain)
|
||||
--build_executable('run_tests', unit_tests, toolchain)
|
||||
end
|
||||
@@ -0,0 +1,140 @@
|
||||
|
||||
#include <stdint.h>
|
||||
#include <stdlib.h>
|
||||
#include <limits.h>
|
||||
#include <stdio.h>
|
||||
|
||||
//#define DEBUG_PROTOCOL
|
||||
void hexdump(const uint8_t* buf, size_t len);
|
||||
|
||||
#include <fibre/crc.hpp>
|
||||
#include <fibre/decoders.hpp>
|
||||
#include <fibre/encoders.hpp>
|
||||
|
||||
void hexdump(const uint8_t* buf, size_t len) {
|
||||
for (size_t pos = 0; pos < len; ++pos) {
|
||||
printf(" %02x", buf[pos]);
|
||||
if ((((pos + 1) % 16) == 0) || ((pos + 1) == len))
|
||||
printf("\r\n");
|
||||
//osDelay(2);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
bool varint_decoder_test() {
|
||||
struct test_case_t {
|
||||
uint8_t encoded[10];
|
||||
size_t length;
|
||||
uint32_t decoded;
|
||||
};
|
||||
const test_case_t test_cases[] = {
|
||||
// encoded, length, decoded
|
||||
{ { 0x00 }, 1, 0 },
|
||||
{ { 0x01 }, 1, 1 },
|
||||
{ { 0xff, 0x01 }, 2, 0xff },
|
||||
{ { 0xAC, 0x02 }, 2, 300 },
|
||||
{ { 0xff, 0xff, 0xff, 0xff, 0xf }, 5, 0xffffffff }
|
||||
};
|
||||
|
||||
for (size_t i = 0; i < sizeof(test_cases) / sizeof(test_cases[0]); ++i) {
|
||||
uint32_t result;
|
||||
VarintStreamDecoder<uint32_t> decoder = make_varint_decoder(result);
|
||||
size_t processed_bytes = 0;
|
||||
int status = decoder.process_bytes(test_cases[i].encoded, test_cases[i].length, &processed_bytes);
|
||||
if (status) {
|
||||
return false;
|
||||
} else if (processed_bytes != test_cases[i].length) {
|
||||
printf("test %zu: expected to process %zu bytes but processed %zu bytes\n", i, test_cases[i].length, processed_bytes);
|
||||
return false;
|
||||
} else if (result != test_cases[i].decoded) {
|
||||
printf("test %zu: expected %u but got %u\n", i, test_cases[i].decoded, result);
|
||||
return false;
|
||||
}
|
||||
|
||||
VarintStreamEncoder<uint32_t> encoder = make_varint_encoder(test_cases[i].decoded);
|
||||
uint8_t buffer[10];
|
||||
size_t generated_bytes = 0;
|
||||
status = encoder.get_bytes(buffer, sizeof(buffer), &generated_bytes);
|
||||
if (status) {
|
||||
return false;
|
||||
} else if ((generated_bytes != test_cases[i].length)
|
||||
|| memcmp(buffer, test_cases[i].encoded, test_cases[i].length)) {
|
||||
printf("test %zu: expected:", i);
|
||||
hexdump(test_cases[i].encoded, test_cases[i].length);
|
||||
printf("got: ");
|
||||
hexdump(buffer, generated_bytes);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
|
||||
int main(void) {
|
||||
/***** Decoder demo (remove or move somewhere else) *****/
|
||||
printf("Running decoder... ");
|
||||
// prepare raw data
|
||||
uint8_t raw_data[] = { 0xBC, 0x03, 0xAC, 0x5e, 0x02, 0x00, 0x00, 0xd1 };
|
||||
//raw_data[3] = calc_crc8<CANONICAL_CRC8_POLYNOMIAL>(CANONICAL_CRC8_INIT, raw_data, 3);
|
||||
//raw_data[7] = calc_crc8<CANONICAL_CRC8_POLYNOMIAL>(raw_data[3], raw_data + 4, 3);
|
||||
|
||||
// instantiate decoder
|
||||
ReceiverState state;
|
||||
auto decoder = make_crc8_decoder<CANONICAL_CRC8_INIT, CANONICAL_CRC8_POLYNOMIAL>(
|
||||
make_decoder_chain(
|
||||
make_length_decoder(state),
|
||||
make_endpoint_id_decoder(state)
|
||||
)
|
||||
);
|
||||
|
||||
// push the raw data through the decoder
|
||||
size_t processed_bytes = 0;
|
||||
int status = decoder.process_bytes(raw_data, sizeof(raw_data), &processed_bytes);
|
||||
|
||||
// expected result: "length: 444, endpoint-id: 300, processed 8 bytes"
|
||||
if (status == 0)
|
||||
printf("length: %zu, endpoint-id: %zu, processed %zu bytes\n", state.length, state.endpoint_id, processed_bytes);
|
||||
else
|
||||
printf("decoder demo failed\n");
|
||||
|
||||
|
||||
/***** Encoder demo (remove or move somewhere else) *****/
|
||||
printf("Running encoder... ");
|
||||
// prepare request
|
||||
Request request = {
|
||||
.endpoint_id = 300,
|
||||
.length = 444,
|
||||
};
|
||||
|
||||
// construct encoder for the request
|
||||
auto e2 = make_crc8_encoder<CANONICAL_CRC8_INIT, CANONICAL_CRC8_POLYNOMIAL>(
|
||||
make_encoder_chain(
|
||||
make_length_encoder(request),
|
||||
make_endpoint_id_encoder(request)
|
||||
)
|
||||
);
|
||||
|
||||
// pull raw data out of the encoder
|
||||
uint8_t buffer[20];
|
||||
size_t generated_bytes = 0;
|
||||
status = e2.get_bytes(buffer, sizeof(buffer), &generated_bytes);
|
||||
if (status == 0) {
|
||||
printf("generated %zu bytes:\n", generated_bytes);
|
||||
hexdump(buffer, generated_bytes);
|
||||
} else {
|
||||
printf("encoder demo failed\n");
|
||||
}
|
||||
|
||||
|
||||
/***** run automated test *****/
|
||||
bool test_result = varint_decoder_test();
|
||||
if (test_result) {
|
||||
printf("all tests passed\n");
|
||||
return 0;
|
||||
} else {
|
||||
printf("some tests failed\n");
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
|
||||
#include <stdio.h>
|
||||
#include <unistd.h>
|
||||
#include <thread>
|
||||
#include <signal.h>
|
||||
|
||||
#include <fibre/protocol.hpp>
|
||||
#include <fibre/posix_tcp.hpp>
|
||||
#include <fibre/posix_udp.hpp>
|
||||
|
||||
|
||||
class TestClass {
|
||||
public:
|
||||
float property1;
|
||||
float property2;
|
||||
|
||||
float set_both(float arg1, float arg2) {
|
||||
property1 = arg1;
|
||||
property2 = arg2;
|
||||
return property1 + property2;
|
||||
}
|
||||
|
||||
FIBRE_EXPORTS(TestClass,
|
||||
make_protocol_property("property1", &property1),
|
||||
make_protocol_property("property2", &property2),
|
||||
make_protocol_function("set_both", *obj, &TestClass::set_both, "arg1", "arg2")
|
||||
);
|
||||
};
|
||||
|
||||
|
||||
int main() {
|
||||
printf("Starting Fibre server...\n");
|
||||
|
||||
TestClass test_object = TestClass();
|
||||
|
||||
// publish the object on Fibre
|
||||
auto definitions = test_object.fibre_definitions;
|
||||
fibre_publish(definitions);
|
||||
|
||||
// Expose Fibre objects on TCP and UDP
|
||||
std::thread server_thread_tcp(serve_on_tcp, 9910);
|
||||
std::thread server_thread_udp(serve_on_udp, 9910);
|
||||
printf("Fibre server started.\n");
|
||||
|
||||
// Dump property1 value
|
||||
while (1) {
|
||||
printf("test_object.property1: %f\n", test_object.property1);
|
||||
usleep(1000000 / 5); // 5 Hz
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Connect to a Fibre-enabled device to play with in the IPython interactive shell.
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
import os
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.realpath(__file__))) + "/python")
|
||||
|
||||
from fibre import Logger, Event
|
||||
|
||||
# Parse arguments
|
||||
parser = argparse.ArgumentParser(description='Connect to a fibre-enabled device to play with it in the IPython interactive shell.')
|
||||
parser.add_argument("-p", "--path", metavar="PATH", action="store",
|
||||
help="The path(s) where ODrive(s) should be discovered.\n"
|
||||
"By default the script will connect to any ODrive on USB.\n\n"
|
||||
"To select a specific USB device:\n"
|
||||
" --path usb:BUS:DEVICE\n"
|
||||
"usbwhere BUS and DEVICE are the bus and device numbers as shown in `lsusb`.\n\n"
|
||||
"To select a specific serial port:\n"
|
||||
" --path serial:PATH\n"
|
||||
"where PATH is the path of the serial port. For example \"/dev/ttyUSB0\".\n"
|
||||
"You can use `ls /dev/tty*` to find the correct port.\n\n"
|
||||
"You can combine USB and serial specs by separating them with a comma (no space!)\n"
|
||||
"Example:\n"
|
||||
" --path usb,serial:/dev/ttyUSB0\n"
|
||||
"means \"discover any USB device or a serial device on /dev/ttyUSB0\"")
|
||||
parser.add_argument("-s", "--serial-number", action="store",
|
||||
help="The 12-digit serial number of the device. "
|
||||
"This is a string consisting of 12 upper case hexadecimal "
|
||||
"digits as displayed in lsusb. \n"
|
||||
" example: 385F324D3037\n"
|
||||
"You can list all devices connected to USB by running\n"
|
||||
"(lsusb -d 1209:0d32 -v; lsusb -d 0483:df11 -v) | grep iSerial\n"
|
||||
"If omitted, any device is accepted.")
|
||||
parser.add_argument("--no-ipython", action="store_true",
|
||||
help="Use the regular Python shell "
|
||||
"instead of the IPython shell, "
|
||||
"even if IPython is installed.")
|
||||
parser.add_argument("-v", "--verbose", action="store_true",
|
||||
help="print debug information")
|
||||
|
||||
parser.set_defaults(path="usb,tcp:localhost:9910")
|
||||
args = parser.parse_args()
|
||||
|
||||
logger = Logger(verbose=args.verbose)
|
||||
app_shutdown_token = Event()
|
||||
|
||||
def print_banner():
|
||||
pass
|
||||
|
||||
def print_help(args, have_devices):
|
||||
pass
|
||||
|
||||
import fibre
|
||||
fibre.launch_shell(args, {}, print_banner, print_help, logger, app_shutdown_token)
|
||||
@@ -0,0 +1,685 @@
|
||||
#!/bin/python3
|
||||
|
||||
import yaml
|
||||
import json
|
||||
import jinja2
|
||||
import jsonschema
|
||||
import re
|
||||
import argparse
|
||||
import sys
|
||||
from collections import OrderedDict
|
||||
|
||||
# This schema describes what we expect interface definition files to look like
|
||||
validator = jsonschema.Draft4Validator(yaml.safe_load("""
|
||||
definitions:
|
||||
interface:
|
||||
type: object
|
||||
properties:
|
||||
c_is_class: {type: boolean}
|
||||
c_name: {type: string}
|
||||
brief: {type: string}
|
||||
doc: {type: string}
|
||||
functions:
|
||||
type: object
|
||||
additionalProperties: {"$ref": "#/definitions/function"}
|
||||
attributes:
|
||||
type: object
|
||||
additionalProperties: {"$ref": "#/definitions/attribute"}
|
||||
__line__: {type: object}
|
||||
__column__: {type: object}
|
||||
required: [c_is_class]
|
||||
additionalProperties: false
|
||||
|
||||
valuetype:
|
||||
type: object
|
||||
properties:
|
||||
mode: {type: string} # this shouldn't be here
|
||||
c_name: {type: string}
|
||||
values: {type: object}
|
||||
flags: {type: object}
|
||||
nullflag: {type: string}
|
||||
__line__: {type: object}
|
||||
__column__: {type: object}
|
||||
additionalProperties: false
|
||||
|
||||
intf_or_val_type:
|
||||
anyOf:
|
||||
- {"$ref": "#/definitions/interface"}
|
||||
- {"$ref": "#/definitions/valuetype"}
|
||||
- {"type": "string"}
|
||||
|
||||
attribute:
|
||||
anyOf: # this is probably not being used correctly
|
||||
- {"$ref": "#/definitions/intf_or_val_type"}
|
||||
- type: object
|
||||
- type: object
|
||||
properties:
|
||||
type: {"$ref": "#/definitions/intf_or_val_type"}
|
||||
c_name: {"type": string}
|
||||
unit: {"type": string}
|
||||
doc: {"type": string}
|
||||
additionalProperties: false
|
||||
|
||||
function:
|
||||
anyOf:
|
||||
- type: 'null'
|
||||
- type: object
|
||||
properties:
|
||||
in: {type: object}
|
||||
out: {type: object}
|
||||
brief: {type: string}
|
||||
doc: {type: string}
|
||||
__line__: {type: object}
|
||||
__column__: {type: object}
|
||||
additionalProperties: false
|
||||
|
||||
|
||||
type: object
|
||||
properties:
|
||||
ns: {type: string}
|
||||
version: {type: string}
|
||||
summary: {type: string}
|
||||
dictionary: {type: array, items: {type: string}}
|
||||
interfaces:
|
||||
type: object
|
||||
additionalProperties: { "$ref": "#/definitions/interface" }
|
||||
valuetypes:
|
||||
type: object
|
||||
additionalProperties: { "$ref": "#/definitions/valuetype" }
|
||||
__line__: {type: object}
|
||||
__column__: {type: object}
|
||||
additionalProperties: false
|
||||
"""))
|
||||
|
||||
# Source: https://stackoverflow.com/a/53647080/3621512
|
||||
class SafeLineLoader(yaml.SafeLoader):
|
||||
pass
|
||||
# def compose_node(self, parent, index):
|
||||
# # the line number where the previous token has ended (plus empty lines)
|
||||
# line = self.line
|
||||
# node = super(SafeLineLoader, self).compose_node(parent, index)
|
||||
# node.__line__ = line + 1
|
||||
# return node
|
||||
#
|
||||
# def construct_mapping(self, node, deep=False):
|
||||
# mapping = super(SafeLineLoader, self).construct_mapping(node, deep=deep)
|
||||
# mapping['__line__'] = node.__line__
|
||||
# #mapping['__column__'] = node.start_mark.column + 1
|
||||
# return mapping
|
||||
|
||||
# Ensure that dicts remain ordered, even in Python <3.6
|
||||
# source: https://stackoverflow.com/a/21912744/3621512
|
||||
def construct_mapping(loader, node):
|
||||
loader.flatten_mapping(node)
|
||||
return OrderedDict(loader.construct_pairs(node))
|
||||
SafeLineLoader.add_constructor(yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, construct_mapping)
|
||||
|
||||
dictionary = []
|
||||
|
||||
def get_words(string):
|
||||
"""
|
||||
Splits a string in PascalCase into a list of lower case words
|
||||
"""
|
||||
regex = ''.join((re.escape(w) + '|') for w in dictionary) + '[a-z0-9]+|[A-Z][a-z0-9]*'
|
||||
return [(w if w in dictionary else w.lower()) for w in re.findall(regex, string)]
|
||||
|
||||
def join_name(*names, delimiter: str = '.'):
|
||||
"""
|
||||
Joins two name components.
|
||||
e.g. 'io.helloworld' + 'sayhello' => 'io.helloworld.sayhello'
|
||||
"""
|
||||
return delimiter.join(y for x in names for y in x.split(delimiter) if y != '')
|
||||
|
||||
def split_name(name, delimiter: str = '.'):
|
||||
def replace_delimiter_in_parentheses():
|
||||
parenthesis_depth = 0
|
||||
for c in name:
|
||||
parenthesis_depth += 1 if c == '<' else -1 if c == '>' else 0
|
||||
yield c if (parenthesis_depth == 0) or (c != delimiter) else ':'
|
||||
return [part.replace(':', '.') for part in ''.join(replace_delimiter_in_parentheses()).split('.')]
|
||||
|
||||
def to_pascal_case(s): return ''.join([(w.title() if not w in dictionary else w) for w in get_words(s)])
|
||||
def to_camel_case(s): return ''.join([(c.lower() if i == 0 else c) for i, c in enumerate(''.join([w.title() for w in get_words(s)]))])
|
||||
def to_macro_case(s): return '_'.join(get_words(s)).upper()
|
||||
def to_snake_case(s): return '_'.join(get_words(s)).lower()
|
||||
def to_kebab_case(s): return '-'.join(get_words(s)).lower()
|
||||
|
||||
value_types = OrderedDict({
|
||||
'bool': {'builtin': True, 'fullname': 'bool', 'name': 'bool', 'c_name': 'bool', 'py_type': 'bool'},
|
||||
'float32': {'builtin': True, 'fullname': 'float32', 'name': 'float32', 'c_name': 'float', 'py_type': 'float'},
|
||||
'uint8': {'builtin': True, 'fullname': 'uint8', 'name': 'uint8', 'c_name': 'uint8_t', 'py_type': 'int'},
|
||||
'uint16': {'builtin': True, 'fullname': 'uint16', 'name': 'uint16', 'c_name': 'uint16_t', 'py_type': 'int'},
|
||||
'uint32': {'builtin': True, 'fullname': 'uint32', 'name': 'uint32', 'c_name': 'uint32_t', 'py_type': 'int'},
|
||||
'uint64': {'builtin': True, 'fullname': 'uint64', 'name': 'uint64', 'c_name': 'uint64_t', 'py_type': 'int'},
|
||||
'int8': {'builtin': True, 'fullname': 'int8', 'name': 'int8', 'c_name': 'int8_t', 'py_type': 'int'},
|
||||
'int16': {'builtin': True, 'fullname': 'int16', 'name': 'int16', 'c_name': 'int16_t', 'py_type': 'int'},
|
||||
'int32': {'builtin': True, 'fullname': 'int32', 'name': 'int32', 'c_name': 'int32_t', 'py_type': 'int'},
|
||||
'int64': {'builtin': True, 'fullname': 'int64', 'name': 'int64', 'c_name': 'int64_t', 'py_type': 'int'},
|
||||
'endpoint_ref': {'builtin': True, 'fullname': 'endpoint_ref', 'name': 'endpoint_ref', 'c_name': 'endpoint_ref_t', 'py_type': '[not implemented]'},
|
||||
})
|
||||
|
||||
enums = OrderedDict()
|
||||
|
||||
interfaces = OrderedDict()
|
||||
|
||||
def make_property_type(typeargs):
|
||||
value_type = resolve_valuetype('', typeargs['fibre.Property.type'])
|
||||
mode = typeargs.get('fibre.Property.mode', 'readwrite')
|
||||
name = 'Property<' + value_type['fullname'] + ', ' + mode + '>'
|
||||
fullname = join_name('fibre', name)
|
||||
if fullname in interfaces:
|
||||
return interfaces[fullname]
|
||||
|
||||
c_name = 'Property<' + ('const ' if mode == 'readonly' else '') + value_type['c_name'] + '>'
|
||||
prop_type = {
|
||||
'name': name,
|
||||
'fullname': fullname,
|
||||
'purename': 'fibre.Property',
|
||||
'c_name': c_name,
|
||||
'value_type': value_type, # TODO: should be a metaarg
|
||||
'mode': mode, # TODO: should be a metaarg
|
||||
'builtin': True,
|
||||
'attributes': OrderedDict(),
|
||||
'functions': OrderedDict()
|
||||
}
|
||||
if mode != 'readonly':
|
||||
prop_type['functions']['exchange'] = {
|
||||
'name': 'exchange',
|
||||
'fullname': join_name(fullname, 'exchange'),
|
||||
'in': OrderedDict([('obj', {'name': 'obj', 'type': {'c_name': c_name}}), ('value', {'name': 'value', 'type': value_type, 'optional': True})]),
|
||||
'out': OrderedDict([('value', {'name': 'value', 'type': value_type})]),
|
||||
#'implementation': 'fibre_property_exchange<' + value_type['c_name'] + '>'
|
||||
}
|
||||
else:
|
||||
prop_type['functions']['read'] = {
|
||||
'name': 'read',
|
||||
'fullname': join_name(fullname, 'read'),
|
||||
'in': OrderedDict([('obj', {'name': 'obj', 'type': {'c_name': c_name}})]),
|
||||
'out': OrderedDict([('value', {'name': 'value', 'type': value_type})]),
|
||||
#'implementation': 'fibre_property_read<' + value_type['c_name'] + '>'
|
||||
}
|
||||
|
||||
interfaces[fullname] = prop_type
|
||||
return prop_type
|
||||
|
||||
generics = {
|
||||
'fibre.Property': make_property_type # TODO: improve generic support
|
||||
}
|
||||
|
||||
|
||||
def make_ref_type(interface):
|
||||
name = 'Ref<' + interface['fullname'] + '>'
|
||||
fullname = join_name('fibre', name)
|
||||
if fullname in interfaces:
|
||||
return interfaces[fullname]
|
||||
|
||||
ref_type = {
|
||||
'builtin': True,
|
||||
'name': name,
|
||||
'fullname': fullname,
|
||||
'c_name': interface['fullname'].replace('.', 'Intf::') + 'Intf*'
|
||||
}
|
||||
value_types[fullname] = ref_type
|
||||
|
||||
return ref_type
|
||||
|
||||
def get_dict(elem, key):
|
||||
return elem.get(key, None) or OrderedDict()
|
||||
|
||||
def regularize_arg(path, name, elem):
|
||||
if elem is None:
|
||||
elem = {}
|
||||
elif isinstance(elem, str):
|
||||
elem = {'type': elem}
|
||||
elem['name'] = name
|
||||
elem['fullname'] = path = join_name(path, name)
|
||||
elem['type'] = regularize_valuetype(path, name, elem['type'])
|
||||
return elem
|
||||
|
||||
def regularize_func(path, name, elem, prepend_args):
|
||||
if elem is None:
|
||||
elem = {}
|
||||
elem['name'] = name
|
||||
elem['fullname'] = path = join_name(path, name)
|
||||
elem['in'] = OrderedDict((n, regularize_arg(path, n, arg))
|
||||
for n, arg in (*prepend_args.items(), *get_dict(elem, 'in').items()))
|
||||
elem['out'] = OrderedDict((n, regularize_arg(path, n, arg))
|
||||
for n, arg in get_dict(elem, 'out').items())
|
||||
return elem
|
||||
|
||||
def regularize_attribute(parent, name, elem, c_is_class):
|
||||
if elem is None:
|
||||
elem = {}
|
||||
if isinstance(elem, str):
|
||||
elem = {'type': elem}
|
||||
elif not 'type' in elem:
|
||||
elem['type'] = {}
|
||||
if 'attributes' in elem: elem['type']['attributes'] = elem.pop('attributes')
|
||||
if 'functions' in elem: elem['type']['functions'] = elem.pop('functions')
|
||||
if 'c_is_class' in elem: elem['type']['c_is_class'] = elem.pop('c_is_class')
|
||||
if 'values' in elem: elem['type']['values'] = elem.pop('values')
|
||||
if 'flags' in elem: elem['type']['flags'] = elem.pop('flags')
|
||||
if 'nullflag' in elem: elem['type']['nullflag'] = elem.pop('nullflag')
|
||||
|
||||
elem['name'] = name
|
||||
elem['fullname'] = join_name(parent['fullname'], name)
|
||||
elem['parent'] = parent
|
||||
elem['typeargs'] = elem.get('typeargs', {})
|
||||
elem['c_name'] = elem.get('c_name', None) or (elem['name'] + ('_' if c_is_class else ''))
|
||||
if ('c_getter' in elem) or ('c_setter' in elem):
|
||||
elem['c_getter'] = elem.get('c_getter', elem['c_name'])
|
||||
elem['c_setter'] = elem.get('c_setter', elem['c_name'] + ' = ')
|
||||
|
||||
if isinstance(elem['type'], str) and elem['type'].startswith('readonly '):
|
||||
elem['typeargs']['fibre.Property.mode'] = 'readonly'
|
||||
elem['typeargs']['fibre.Property.type'] = elem['type'][len('readonly '):]
|
||||
elem['type'] = 'fibre.Property'
|
||||
if elem['typeargs']['fibre.Property.mode'] == 'readonly' and 'c_setter' in elem: elem.pop('c_setter')
|
||||
elif ('flags' in elem['type']) or ('values' in elem['type']):
|
||||
elem['typeargs']['fibre.Property.mode'] = elem['typeargs'].get('fibre.Property.mode', None) or 'readwrite'
|
||||
elem['typeargs']['fibre.Property.type'] = regularize_valuetype(parent['fullname'], to_pascal_case(name), elem['type'])
|
||||
elem['type'] = 'fibre.Property'
|
||||
if elem['typeargs']['fibre.Property.mode'] == 'readonly' and 'c_setter' in elem: elem.pop('c_setter')
|
||||
else:
|
||||
elem['type'] = regularize_interface(parent['fullname'], to_pascal_case(name), elem['type'])
|
||||
return elem
|
||||
|
||||
|
||||
def regularize_interface(path, name, elem):
|
||||
if elem is None:
|
||||
elem = {}
|
||||
if isinstance(elem, str):
|
||||
return elem # will be resolved during type resolution
|
||||
#if path is None:
|
||||
# max_anonymous_type = max([int((re.findall('^' + join_name(path, 'AnonymousType') + '([1-9]+)$', x) + ['0'])[0]) for x in interfaces.keys()])
|
||||
# path = 'AnonymousType' + str(max_anonymous_type + 1)
|
||||
elem['name'] = split_name(name)[-1]
|
||||
elem['fullname'] = path = join_name(path, name)
|
||||
elem['c_name'] = elem.get('c_name', elem['fullname'].replace('.', 'Intf::')) + 'Intf'
|
||||
interfaces[path] = elem
|
||||
elem['functions'] = OrderedDict((name, regularize_func(path, name, func, {'obj': {'type': make_ref_type(elem)}}))
|
||||
for name, func in get_dict(elem, 'functions').items())
|
||||
if not 'c_is_class' in elem:
|
||||
raise Exception(elem)
|
||||
treat_as_class = elem['c_is_class'] # TODO: add command line arg to make this selectively optional
|
||||
elem['attributes'] = OrderedDict((name, regularize_attribute(elem, name, prop, treat_as_class))
|
||||
for name, prop in get_dict(elem, 'attributes').items())
|
||||
elem['interfaces'] = []
|
||||
elem['enums'] = []
|
||||
return elem
|
||||
|
||||
def regularize_valuetype(path, name, elem):
|
||||
if elem is None:
|
||||
elem = {}
|
||||
if isinstance(elem, str):
|
||||
return elem # will be resolved during type resolution
|
||||
elem['name'] = split_name(name)[-1]
|
||||
elem['fullname'] = path = join_name(path, name)
|
||||
elem['c_name'] = elem.get('c_name', elem['fullname'].replace('.', 'Intf::'))
|
||||
value_types[path] = elem
|
||||
|
||||
if 'flags' in elem: # treat as flags
|
||||
bit = 0
|
||||
for k, v in elem['flags'].items():
|
||||
elem['flags'][k] = elem['flags'][k] or OrderedDict()
|
||||
elem['flags'][k]['name'] = k
|
||||
current_bit = elem['flags'][k].get('bit', bit)
|
||||
elem['flags'][k]['bit'] = current_bit
|
||||
elem['flags'][k]['value'] = 0 if current_bit is None else (1 << current_bit)
|
||||
bit = bit if current_bit is None else current_bit + 1
|
||||
if 'nullflag' in elem:
|
||||
elem['flags'] = OrderedDict([(elem['nullflag'], {'value': 0, 'bit': None}), *elem['flags'].items()])
|
||||
elem['values'] = elem['flags']
|
||||
elem['is_flags'] = True
|
||||
elem['is_enum'] = True
|
||||
enums[path] = elem
|
||||
|
||||
elif 'values' in elem: # treat as enum
|
||||
val = 0
|
||||
for k, v in elem['values'].items():
|
||||
elem['values'][k] = elem['values'][k] or OrderedDict()
|
||||
elem['values'][k]['name'] = k
|
||||
val = elem['values'][k].get('value', val)
|
||||
elem['values'][k]['value'] = val
|
||||
val += 1
|
||||
enums[path] = elem
|
||||
elem['is_enum'] = True
|
||||
|
||||
return elem
|
||||
|
||||
def resolve_interface(scope, name, typeargs):
|
||||
"""
|
||||
Resolves a type name (i.e. interface name or value type name) given as a
|
||||
string to an interface object. The innermost scope is searched first.
|
||||
At every scope level, if no matching interface is found, it is checked if a
|
||||
matching value type exists. If so, the interface type fibre.Property<value_type>
|
||||
is returned.
|
||||
"""
|
||||
if not isinstance(name, str):
|
||||
return name
|
||||
|
||||
if 'fibre.Property.type' in typeargs:
|
||||
typeargs['fibre.Property.type'] = resolve_valuetype(scope, typeargs['fibre.Property.type'])
|
||||
|
||||
scope = scope.split('.')
|
||||
for probe_scope in [join_name(*scope[:(len(scope)-i)]) for i in range(len(scope)+1)]:
|
||||
probe_name = join_name(probe_scope, name)
|
||||
#print('probing ' + probe_name)
|
||||
if probe_name in interfaces:
|
||||
return interfaces[probe_name]
|
||||
elif probe_name in value_types:
|
||||
typeargs['fibre.Property.type'] = value_types[probe_name]
|
||||
return make_property_type(typeargs)
|
||||
elif probe_name in generics:
|
||||
return generics[probe_name](typeargs)
|
||||
|
||||
raise Exception('could not resolve type {} in {}. Known interfaces are: {}. Known value types are: {}'.format(name, join_name(*scope), list(interfaces.keys()), list(value_types.keys())))
|
||||
|
||||
def resolve_valuetype(scope, name):
|
||||
"""
|
||||
Resolves a type name given as a string to the type object.
|
||||
The innermost scope is searched first.
|
||||
"""
|
||||
if not isinstance(name, str):
|
||||
return name
|
||||
|
||||
scope = scope.split('.')
|
||||
for probe_scope in [join_name(*scope[:(len(scope)-i)]) for i in range(len(scope)+1)]:
|
||||
probe_name = join_name(probe_scope, name)
|
||||
if probe_name in value_types:
|
||||
return value_types[probe_name]
|
||||
|
||||
raise Exception('could not resolve type {} in {}. Known value types are: {}'.format(name, join_name(*scope), list(value_types.keys())))
|
||||
|
||||
|
||||
def map_to_fibre01_type(t):
|
||||
if t.get('is_enum', False):
|
||||
return 'int32'
|
||||
elif t['fullname'] == 'float32':
|
||||
return 'float'
|
||||
return t['fullname']
|
||||
|
||||
def generate_endpoint_for_property(prop, attr_bindto, idx):
|
||||
prop_intf = interfaces[prop['type']['fullname']]
|
||||
|
||||
endpoint = {
|
||||
'id': idx,
|
||||
'function': prop_intf['functions']['read' if prop['type']['mode'] == 'readonly' else 'exchange'],
|
||||
'in_bindings': OrderedDict([('obj', attr_bindto)]),
|
||||
'out_bindings': OrderedDict()
|
||||
}
|
||||
endpoint_definition = {
|
||||
'name': prop['name'],
|
||||
'id': idx,
|
||||
'type': map_to_fibre01_type(prop['type']['value_type']),
|
||||
'access': 'r' if prop['type']['mode'] == 'readonly' else 'rw',
|
||||
}
|
||||
return endpoint, endpoint_definition
|
||||
|
||||
def generate_endpoint_table(intf, bindto, idx):
|
||||
"""
|
||||
Generates a Fibre v0.1 endpoint table for a given interface.
|
||||
This will probably be deprecated in the future.
|
||||
The object must have no circular property types (i.e. A.b has type B and B.a has type A).
|
||||
"""
|
||||
endpoints = []
|
||||
endpoint_definitions = []
|
||||
cnt = 0
|
||||
|
||||
for k, prop in intf['attributes'].items():
|
||||
property_value_type = re.findall('^fibre\.Property<([^>]*), (readonly|readwrite)>$', prop['type']['fullname'])
|
||||
#attr_bindto = join_name(bindto, bindings_map.get(join_name(intf['fullname'], k), k + ('_' if len(intf['functions']) or (intf['fullname'] in treat_as_classes) else '')))
|
||||
attr_bindto = intf['c_name'] + '::get_' + prop['name'] + '(' + bindto + ')'
|
||||
if len(property_value_type):
|
||||
# Special handling for Property<...> attributes: they resolve to one single endpoint
|
||||
endpoint, endpoint_definition = generate_endpoint_for_property(prop, attr_bindto, idx + cnt)
|
||||
endpoints.append(endpoint)
|
||||
endpoint_definitions.append(endpoint_definition)
|
||||
cnt += 1
|
||||
else:
|
||||
inner_endpoints, inner_endpoint_definitions, inner_cnt = generate_endpoint_table(prop['type'], attr_bindto, idx + cnt)
|
||||
endpoints += inner_endpoints
|
||||
endpoint_definitions.append({
|
||||
'name': k,
|
||||
'type': 'object',
|
||||
'members': inner_endpoint_definitions
|
||||
})
|
||||
cnt += inner_cnt
|
||||
|
||||
for k, func in intf['functions'].items():
|
||||
endpoints.append({
|
||||
'id': idx + cnt,
|
||||
'function': func,
|
||||
'in_bindings': OrderedDict([('obj', bindto), *[(k_arg, '(' + bindto + ')->' + func['name'] + '_in_' + k_arg + '_') for k_arg in list(func['in'].keys())[1:]]]),
|
||||
'out_bindings': OrderedDict((k_arg, '&(' + bindto + ')->' + func['name'] + '_out_' + k_arg + '_') for k_arg in func['out'].keys()),
|
||||
})
|
||||
in_def = []
|
||||
out_def = []
|
||||
for i, (k_arg, arg) in enumerate(list(func['in'].items())[1:]):
|
||||
endpoint, endpoint_definition = generate_endpoint_for_property({
|
||||
'name': arg['name'],
|
||||
'type': make_property_type({'fibre.Property.type': arg['type'], 'fibre.Property.mode': 'readwrite'})
|
||||
}, intf['c_name'] + '::get_' + func['name'] + '_in_' + k_arg + '_' + '(' + bindto + ')', idx + cnt + 1 + i)
|
||||
endpoints.append(endpoint)
|
||||
in_def.append(endpoint_definition)
|
||||
for i, (k_arg, arg) in enumerate(func['out'].items()):
|
||||
endpoint, endpoint_definition = generate_endpoint_for_property({
|
||||
'name': arg['name'],
|
||||
'type': make_property_type({'fibre.Property.type': arg['type'], 'fibre.Property.mode': 'readonly'})
|
||||
}, intf['c_name'] + '::get_' + func['name'] + '_out_' + k_arg + '_' + '(' + bindto + ')', idx + cnt + len(func['in']) + i)
|
||||
endpoints.append(endpoint)
|
||||
out_def.append(endpoint_definition)
|
||||
|
||||
endpoint_definitions.append({
|
||||
'name': k,
|
||||
'id': idx + cnt,
|
||||
'type': 'function',
|
||||
'inputs': in_def,
|
||||
'outputs': out_def
|
||||
})
|
||||
cnt += len(func['in']) + len(func['out'])
|
||||
|
||||
return endpoints, endpoint_definitions, cnt
|
||||
|
||||
|
||||
# Parse arguments
|
||||
|
||||
parser = argparse.ArgumentParser(description="Gernerate code from YAML interface definitions")
|
||||
parser.add_argument("--version", action="store_true",
|
||||
help="print version information")
|
||||
parser.add_argument("-v", "--verbose", action="store_true",
|
||||
help="print debug information (on stderr)")
|
||||
parser.add_argument("-d", "--definitions", type=argparse.FileType('r', encoding='utf-8'), nargs='+',
|
||||
help="the YAML interface definition file(s) used to generate the code")
|
||||
parser.add_argument("-t", "--template", type=argparse.FileType('r', encoding='utf-8'),
|
||||
help="the code template")
|
||||
group = parser.add_mutually_exclusive_group(required=True)
|
||||
group.add_argument("-o", "--output", type=argparse.FileType('w', encoding='utf-8'),
|
||||
help="path of the generated output")
|
||||
group.add_argument("--outputs", type=str,
|
||||
help="path pattern for the generated outputs. One output is generated for each interface. Use # as placeholder for the interface name.")
|
||||
parser.add_argument("--generate-endpoints", type=str, nargs='?',
|
||||
help="if specified, an endpoint table will be generated and passed to the template for the specified interface")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.version:
|
||||
print("0.0.1")
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
definition_files = args.definitions
|
||||
template_file = args.template
|
||||
|
||||
|
||||
# Load definition files
|
||||
|
||||
for definition_file in definition_files:
|
||||
try:
|
||||
file_content = yaml.load(definition_file, Loader=SafeLineLoader)
|
||||
except yaml.scanner.ScannerError as ex:
|
||||
print("YAML parsing error: " + str(ex), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
for err in validator.iter_errors(file_content):
|
||||
if '__line__' in err.absolute_path:
|
||||
continue
|
||||
if '__column__' in err.absolute_path:
|
||||
continue
|
||||
#instance = err.instance.get(re.findall("([^']*)' (?:was|were) unexpected\)", err.message)[0], err.instance)
|
||||
# TODO: print line number
|
||||
raise Exception(err.message + '\nat ' + str(list(err.absolute_path)))
|
||||
interfaces.update(get_dict(file_content, 'interfaces'))
|
||||
value_types.update(get_dict(file_content, 'valuetypes'))
|
||||
dictionary += file_content.get('dictionary', None) or []
|
||||
|
||||
|
||||
# Preprocess definitions
|
||||
|
||||
# Regularize everything into a wellknown form
|
||||
for k, item in list(interfaces.items()):
|
||||
regularize_interface('', k, item)
|
||||
for k, item in list(value_types.items()):
|
||||
regularize_valuetype('', k, item)
|
||||
|
||||
if args.verbose:
|
||||
print('Known interfaces: ' + ''.join([('\n ' + k) for k in interfaces.keys()]))
|
||||
print('Known value types: ' + ''.join([('\n ' + k) for k in value_types.keys()]))
|
||||
|
||||
clashing_names = list(set(value_types.keys()).intersection(set(interfaces.keys())))
|
||||
if len(clashing_names):
|
||||
print("**Error**: Found both an interface and a value type with the name {}. This is not allowed, interfaces and value types (such as enums) share the same namespace.".format(clashing_names[0]), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Resolve all types into references
|
||||
for _, item in list(interfaces.items()):
|
||||
for _, prop in item['attributes'].items():
|
||||
prop['type'] = resolve_interface(item['fullname'], prop['type'], prop['typeargs'])
|
||||
for _, func in item['functions'].items():
|
||||
for _, arg in func['in'].items():
|
||||
arg['type'] = resolve_valuetype(item['fullname'], arg['type'])
|
||||
for _, arg in func['out'].items():
|
||||
arg['type'] = resolve_valuetype(item['fullname'], arg['type'])
|
||||
|
||||
# Attach interfaces to their parents
|
||||
toplevel_interfaces = []
|
||||
for k, item in list(interfaces.items()):
|
||||
k = split_name(k)
|
||||
if len(k) == 1:
|
||||
toplevel_interfaces.append(item)
|
||||
else:
|
||||
if k[:-1] != ['fibre']: # TODO: remove special handling
|
||||
parent = interfaces[join_name(*k[:-1])]
|
||||
parent['interfaces'].append(item)
|
||||
item['parent'] = parent
|
||||
toplevel_enums = []
|
||||
for k, item in list(enums.items()):
|
||||
k = split_name(k)
|
||||
if len(k) == 1:
|
||||
toplevel_enums.append(item)
|
||||
else:
|
||||
if k[:-1] != ['fibre']: # TODO: remove special handling
|
||||
parent = interfaces[join_name(*k[:-1])]
|
||||
parent['enums'].append(item)
|
||||
item['parent'] = parent
|
||||
|
||||
|
||||
if args.generate_endpoints:
|
||||
endpoints, embedded_endpoint_definitions, _ = generate_endpoint_table(interfaces[args.generate_endpoints], '&ep_root', 1) # TODO: make user-configurable
|
||||
embedded_endpoint_definitions = [{'name': '', 'id': 0, 'type': 'json', 'access': 'r'}] + embedded_endpoint_definitions
|
||||
endpoints = [{'id': 0, 'function': {'fullname': 'endpoint0_handler', 'in': {}, 'out': {}}, 'bindings': {}}] + endpoints
|
||||
else:
|
||||
embedded_endpoint_definitions = None
|
||||
endpoints = None
|
||||
|
||||
|
||||
# Render template
|
||||
|
||||
env = jinja2.Environment(
|
||||
comment_start_string='[#', comment_end_string='#]',
|
||||
block_start_string='[%', block_end_string='%]',
|
||||
variable_start_string='[[', variable_end_string=']]'
|
||||
)
|
||||
|
||||
def tokenize(text, interface, interface_transform, value_type_transform, attribute_transform):
|
||||
"""
|
||||
Looks for referencable tokens (interface names, value type names or
|
||||
attribute names) in a documentation text and runs them through the provided
|
||||
processing functions.
|
||||
Tokens are detected by enclosing back-ticks (`).
|
||||
|
||||
interface: The interface type object that defines the scope in which the
|
||||
tokens should be detected.
|
||||
interface_transform: A function that takes an interface object as an argument
|
||||
and returns a string.
|
||||
value_type_transform: A function that takes a value type object as an argument
|
||||
and returns a string.
|
||||
attribute_transform: A function that takes the token strin and an attribute
|
||||
object as arguments and returns a string.
|
||||
"""
|
||||
if text is None or isinstance(text, jinja2.runtime.Undefined):
|
||||
return text
|
||||
|
||||
def token_transform(token):
|
||||
token = token.groups()[0]
|
||||
token_list = split_name(token)
|
||||
|
||||
# Check if this is an attribute reference
|
||||
scope = interface
|
||||
attr = None
|
||||
while attr is None and not scope is None:
|
||||
attr_intf = scope
|
||||
for name in token_list:
|
||||
if not name in attr_intf['attributes']:
|
||||
attr = None
|
||||
break
|
||||
attr = attr_intf['attributes'][name]
|
||||
attr_intf = attr['type']
|
||||
scope = scope.get('parent', None)
|
||||
|
||||
if not attr is None:
|
||||
return attribute_transform(token, attr)
|
||||
|
||||
print('Warning: cannot resolve "{}" in {}'.format(token, interface['fullname']))
|
||||
return "`" + token + "`"
|
||||
|
||||
return re.sub(r'`([A-Za-z\._]+)`', token_transform, text)
|
||||
|
||||
env.filters['to_pascal_case'] = to_pascal_case
|
||||
env.filters['to_camel_case'] = to_camel_case
|
||||
env.filters['to_macro_case'] = to_macro_case
|
||||
env.filters['to_snake_case'] = to_snake_case
|
||||
env.filters['to_kebab_case'] = to_kebab_case
|
||||
env.filters['first'] = lambda x: next(iter(x))
|
||||
env.filters['skip_first'] = lambda x: list(x)[1:]
|
||||
env.filters['to_c_string'] = lambda x: '\n'.join(('"' + line.replace('"', '\\"') + '"') for line in json.dumps(x, separators=(',', ':')).replace('{"name"', '\n{"name"').split('\n'))
|
||||
env.filters['tokenize'] = tokenize
|
||||
env.filters['diagonalize'] = lambda lst: [lst[:i + 1] for i in range(len(lst))]
|
||||
|
||||
template = env.from_string(template_file.read())
|
||||
|
||||
template_args = {
|
||||
'interfaces': interfaces,
|
||||
'value_types': value_types,
|
||||
'toplevel_interfaces': toplevel_interfaces,
|
||||
'endpoints': endpoints,
|
||||
'embedded_endpoint_definitions': embedded_endpoint_definitions
|
||||
}
|
||||
|
||||
if not args.output is None:
|
||||
output = template.render(**template_args)
|
||||
args.output.write(output)
|
||||
else:
|
||||
assert('#' in args.outputs)
|
||||
|
||||
for k, intf in interfaces.items():
|
||||
if split_name(k)[0] == 'fibre':
|
||||
continue # TODO: remove special case
|
||||
output = template.render(interface = intf, **template_args)
|
||||
with open(args.outputs.replace('#', k.lower()), 'w', encoding='utf-8') as output_file:
|
||||
output_file.write(output)
|
||||
|
||||
for k, enum in value_types.items():
|
||||
if enum.get('builtin', False) or not enum.get('is_enum', False):
|
||||
continue
|
||||
output = template.render(enum = enum, **template_args)
|
||||
with open(args.outputs.replace('#', k.lower()), 'w', encoding='utf-8') as output_file:
|
||||
output_file.write(output)
|
||||
@@ -0,0 +1,211 @@
|
||||
|
||||
|
||||
function GCCToolchain(prefix, builddir, compiler_flags, linker_flags)
|
||||
|
||||
-- add some default compiler flags
|
||||
compiler_flags += '-fstack-usage'
|
||||
|
||||
local gcc_generic_compiler = function(compiler, compiler_flags, gen_su_file, src, flags, includes, outputs)
|
||||
-- resolve source path
|
||||
src = tostring(src)
|
||||
|
||||
-- convert include list to flags
|
||||
inc_flags = {}
|
||||
for _,inc in pairs(includes) do
|
||||
inc_flags += "-I"..tostring(inc)
|
||||
end
|
||||
|
||||
obj_file = builddir.."/"..src:gsub("/","_")..".o"
|
||||
outputs.object_files += obj_file
|
||||
if gen_su_file then
|
||||
su_file = builddir.."/"..src:gsub("/","_")..".su"
|
||||
extra_outputs = { su_file }
|
||||
outputs.su_files += su_file
|
||||
else
|
||||
extra_outputs = {}
|
||||
end
|
||||
tup.frule{
|
||||
inputs= { tup.getcwd()..'/'..src },
|
||||
command=compiler..' -c %f '..
|
||||
tostring(compiler_flags)..' '.. -- CFLAGS for this compiler
|
||||
tostring(inc_flags)..' '.. -- CFLAGS for this translation unit
|
||||
tostring(flags).. -- CFLAGS for this translation unit
|
||||
' -o %o',
|
||||
outputs={obj_file,extra_outputs=extra_outputs}
|
||||
}
|
||||
end
|
||||
return {
|
||||
compile_c = function(src, flags, includes, outputs) gcc_generic_compiler(prefix..'gcc -std=c99', compiler_flags, true, src, flags, includes, outputs) end,
|
||||
compile_cpp = function(src, flags, includes, outputs) gcc_generic_compiler(prefix..'g++ -std=c++11', compiler_flags, true, src, flags, includes, outputs) end,
|
||||
compile_asm = function(src, flags, includes, outputs) gcc_generic_compiler(prefix..'gcc -x assembler-with-cpp', compiler_flags, false, src, flags, includes, outputs) end,
|
||||
link = function(objects, libs, output_name)
|
||||
-- convert lib list to flags
|
||||
lib_flags = {}
|
||||
for _,inc in pairs(libs) do
|
||||
lib_flags += "-l"..tostring(inc)
|
||||
end
|
||||
|
||||
output_name = builddir..'/'..output_name
|
||||
tup.frule{
|
||||
inputs=objects,
|
||||
command=prefix..'g++ %f '..
|
||||
tostring(linker_flags)..' '..
|
||||
tostring(lib_flags)..' '..
|
||||
'-Wl,-Map=%O.map'..
|
||||
' -o %o',
|
||||
outputs={output_name..'.elf', extra_outputs={output_name..'.map'}}
|
||||
}
|
||||
-- display the size
|
||||
tup.frule{inputs={output_name..'.elf'}, command=prefix..'size %f'}
|
||||
-- create *.hex and *.bin output formats
|
||||
tup.frule{inputs={output_name..'.elf'}, command=prefix..'objcopy -O ihex %f %o', outputs={output_name..'.hex'}}
|
||||
tup.frule{inputs={output_name..'.elf'}, command=prefix..'objcopy -O binary -S %f %o', outputs={output_name..'.bin'}}
|
||||
end
|
||||
}
|
||||
end
|
||||
|
||||
|
||||
function LLVMToolchain(arch, compiler_flags, linker_flags)
|
||||
|
||||
-- add some default compiler flags
|
||||
--compiler_flags += '-march='..arch
|
||||
compiler_flags += '-std=c++14'
|
||||
|
||||
clang_generic_compiler = function(compiler, compiler_flags, src, flags, includes, outputs)
|
||||
-- add includes to CFLAGS
|
||||
for _,inc in pairs(includes) do
|
||||
flags += "-I"..inc
|
||||
end
|
||||
-- todo: vary build directory
|
||||
obj_file="build/"..src:gsub("/","_")..".o"
|
||||
tup.frule{
|
||||
inputs=src,
|
||||
command=compiler..' -c %f '..
|
||||
tostring(compiler_flags)..' '.. -- CFLAGS for this compiler
|
||||
tostring(flags).. -- CFLAGS for this translation unit
|
||||
' -o %o',
|
||||
outputs={obj_file}
|
||||
}
|
||||
outputs.object_files += obj_file
|
||||
end
|
||||
return {
|
||||
compile_c = function(src, flags, includes, outputs) clang_generic_compiler('clang', compiler_flags, src, flags, includes, outputs) end,
|
||||
compile_cpp = function(src, flags, includes, outputs) clang_generic_compiler('clang++', compiler_flags, src, flags, includes, outputs) end,
|
||||
link = function(objects, output_name)
|
||||
tup.frule{
|
||||
inputs=objects,
|
||||
command='clang++ %f '..
|
||||
tostring(linker_flags)..
|
||||
' -o %o',
|
||||
outputs=output_name
|
||||
}
|
||||
end
|
||||
}
|
||||
end
|
||||
|
||||
|
||||
function get_generalized_paths(paths)
|
||||
if paths == nil then
|
||||
return {}
|
||||
else
|
||||
-- TODO: check for string
|
||||
generalized_paths = {}
|
||||
for _,path in pairs(paths) do
|
||||
table.insert(generalized_paths, tup.nodevariable(path))
|
||||
end
|
||||
return generalized_paths
|
||||
end
|
||||
end
|
||||
|
||||
|
||||
-- A package is a collection of source files and associated
|
||||
-- information required to compile those source files.
|
||||
-- pkg.sources: The source files that shall be compiled as
|
||||
-- part of this package
|
||||
-- pkg.private_headers: The include directories that are required
|
||||
-- to compile the source files in this package
|
||||
-- pkg.headers: The include directories that are _exported_
|
||||
-- by this package. These directories are included
|
||||
-- when compiling other packages that import
|
||||
-- this package.
|
||||
-- pkg.packages: The packages that are needed to compile and link this
|
||||
-- package. The public include directories of each imported
|
||||
-- package are passed to the compiler when compiling
|
||||
-- the source files of this package. The object files
|
||||
-- emitted by the imported packages are included when linking
|
||||
-- this package.
|
||||
-- pkg.libs: The libraries that are needed to link this
|
||||
-- package
|
||||
function define_package(pkg)
|
||||
--print('defined package in '..tup.getcwd())
|
||||
pkg.sources = get_generalized_paths(pkg.sources)
|
||||
pkg.objects = get_generalized_paths(pkg.objects)
|
||||
pkg.headers = get_generalized_paths(pkg.headers)
|
||||
pkg.private_headers = get_generalized_paths(private_headers)
|
||||
if pkg.packages == nil then pkg.packages = {} end
|
||||
if pkg.libs == nil then pkg.libs = {} end
|
||||
if pkg.c_flags == nil then pkg.c_flags = {} end
|
||||
if pkg.cpp_flags == nil then pkg.cpp_flags = {} end
|
||||
if pkg.asm_flags == nil then pkg.asm_flags = {} end
|
||||
return pkg
|
||||
end
|
||||
|
||||
|
||||
-- Builds object files from the source files in the specified package
|
||||
function build_objects(pkg, toolchain)
|
||||
all_headers = {}
|
||||
tup.append_table(all_headers, pkg.private_headers)
|
||||
tup.append_table(all_headers, pkg.headers)
|
||||
|
||||
-- add exported header directories of each imported package
|
||||
for _,imported_pkg in pairs(pkg.packages) do
|
||||
tup.append_table(all_headers, imported_pkg.headers)
|
||||
end
|
||||
|
||||
-- compile
|
||||
outputs = {
|
||||
object_files = {}
|
||||
}
|
||||
tup.append_table(outputs.object_files, pkg.objects)
|
||||
|
||||
for _,src in pairs(pkg.sources) do
|
||||
--print("compile "..src)
|
||||
ext = tup.ext(tostring(src))
|
||||
if ext == 'c' then
|
||||
toolchain.compile_c(src, pkg.c_flags, all_headers, outputs)
|
||||
elseif ext == 'cpp' then
|
||||
toolchain.compile_cpp(src, pkg.cpp_flags, all_headers, outputs)
|
||||
elseif ext == 's' or tup.ext(src) == 'asm' then
|
||||
toolchain.compile_asm(src, pkg.asm_flags, all_headers, outputs)
|
||||
else
|
||||
error('unrecognized file ending')
|
||||
end
|
||||
end
|
||||
|
||||
return outputs.object_files
|
||||
end
|
||||
|
||||
function build_executable(name, pkg, toolchain)
|
||||
all_object_files = {}
|
||||
all_libs = {}
|
||||
|
||||
-- TODO: flatten the import hierarchy prior to compiling
|
||||
|
||||
-- build current package
|
||||
tup.append_table(all_object_files, build_objects(pkg, toolchain))
|
||||
tup.append_table(all_libs, pkg.libs)
|
||||
|
||||
-- build imported packages
|
||||
for _,imported_pkg in pairs(pkg.packages) do
|
||||
objects = build_objects(imported_pkg, toolchain)
|
||||
tup.append_table(all_object_files, objects)
|
||||
tup.append_table(all_libs, imported_pkg.libs)
|
||||
end
|
||||
|
||||
-- link
|
||||
--tup.append_table(args.linker_objects, outputs.object_files)
|
||||
print('link objects ')
|
||||
print(all_object_files)
|
||||
print(all_libs)
|
||||
toolchain.link(all_object_files, all_libs, name)
|
||||
end
|
||||
Reference in New Issue
Block a user