QCBOR: Improve handling of end of data and error; add indefinite length encoding
* Minor improvements / fixes in run_test framework
* Add CBOR indefinite length encoding
* Recheck pointer math in UsefulBuf and remove "TODO"
* Better error handling of not-well-formed CBOR when decoding
* Better handling of end of data when decoding
* Better handling of encode error when out of space in output buffer
Change-Id: Ib8dc2af95bc533b7905648d8f8c3b1bf1c42ba44
Signed-off-by: Laurence Lundblade <lgl@securitytheory.com>
diff --git a/lib/ext/qcbor/inc/qcbor.h b/lib/ext/qcbor/inc/qcbor.h
index 2945192..d541a10 100644
--- a/lib/ext/qcbor/inc/qcbor.h
+++ b/lib/ext/qcbor/inc/qcbor.h
@@ -43,6 +43,9 @@
when who what, where, why
-------- ---- ---------------------------------------------------
+ 08/7/19 llundblade Better handling of not well-formed encode and decode.
+ 07/31/19 llundblade New error code for better end of data handling.
+ 7/25/19 janjongboom Add indefinite length encoding for maps and arrays.
05/26/19 llundblade Add QCBOREncode_GetErrorState() and _IsBufferNULL().
04/26/19 llundblade Big documentation & style update. No interface change.
02/16/19 llundblade Redesign MemPool to fix memory access alignment bug.
@@ -217,6 +220,8 @@
#define CBOR_MAJOR_NONE_TYPE_RAW 9
#define CBOR_MAJOR_NONE_TAG_LABEL_REORDER 10
#define CBOR_MAJOR_NONE_TYPE_BSTR_LEN_ONLY 11
+#define CBOR_MAJOR_NONE_TYPE_ARRAY_INDEFINITE_LEN 12
+#define CBOR_MAJOR_NONE_TYPE_MAP_INDEFINITE_LEN 13
/* ===========================================================================
@@ -392,6 +397,8 @@
#define SINGLE_PREC_FLOAT 26
#define DOUBLE_PREC_FLOAT 27
#define CBOR_SIMPLE_BREAK 31
+#define CBOR_SIMPLEV_RESERVED_START CBOR_SIMPLEV_ONEBYTE
+#define CBOR_SIMPLEV_RESERVED_END CBOR_SIMPLE_BREAK
@@ -529,10 +536,12 @@
The encoding error handling is simple. The only possible errors are
trying to encode structures that are too large or too complex. There
are no internal malloc calls so there will be no failures for out of
- memory. Only the final call, QCBOREncode_Finish(), returns an error
- code. Once an error happens, the encoder goes into an error state
- and calls to it will do nothing so the encoding can just go on. An
- error check is not needed after every data item is added.
+ memory. The error state is tracked internally, so there is no need
+ to check for errors when encoding. Only the return code from
+ QCBOREncode_Finish() need be checked as once an error happens, the
+ encoder goes into an error state and calls to it to add more data
+ will do nothing. An error check is not needed after every data item
+ is added.
Encoding generally proceeds by calling QCBOREncode_Init(), calling
lots of @c QCBOREncode_AddXxx() functions and calling
@@ -605,7 +614,6 @@
@ref QCBOR_MAX_ARRAY_NESTING (this is typically 15).
- Max items in an array or map when encoding / decoding is
@ref QCBOR_MAX_ITEMS_IN_ARRAY (typically 65,536).
- - Does not support encoding indefinite lengths (decoding is supported).
- Does not directly support some tagged types: decimal fractions, big floats
- Does not directly support labels in maps other than text strings and integers.
- Does not directly support integer labels greater than @c INT64_MAX.
@@ -679,13 +687,16 @@
/** During decoding, some CBOR construct was encountered that this
decoder doesn't support, primarily this is the reserved
- additional info values, 28 through 30. */
+ additional info values, 28 through 30. During encoding,
+ an attempt to create simple value between 24 and 31. */
QCBOR_ERR_UNSUPPORTED = 5,
/** During decoding, hit the end of the given data to decode. For
example, a byte string of 100 bytes was expected, but the end
of the input was hit before finding those 100 bytes. Corrupted
- CBOR input will often result in this error. */
+ CBOR input will often result in this error. See also @ref
+ QCBOR_ERR_NO_MORE_ITEMS.
+ */
QCBOR_ERR_HIT_END = 6,
/** During encoding, the length of the encoded CBOR exceeded @c
@@ -746,6 +757,16 @@
list, or not enough space in @ref QCBORTagListOut. */
QCBOR_ERR_TOO_MANY_TAGS = 20,
+ /** An integer type is encoded with a bad length (an indefinite length) */
+ QCBOR_ERR_BAD_INT = 21,
+
+ /** All well-formed data items have been consumed and there are no
+ more. If parsing a CBOR stream this indicates the non-error
+ end of the stream. If parsing a CBOR stream / sequence, this
+ probably indicates that some data items expected are not present.
+ See also @ref QCBOR_ERR_HIT_END. */
+ QCBOR_ERR_NO_MORE_ITEMS = 22
+
} QCBORError;
@@ -1781,8 +1802,8 @@
was computed. If a buffer was passed, then the encoded CBOR is in the
buffer.
- All encoding errors manifest here as no other encoding function
- returns any errors. They just set the error state in the encode
+ Encoding errors primarily manifest here as most other encoding function
+ do no return an error. They just set the error state in the encode
context after which no encoding function does anything.
Three types of errors manifest here. The first type are nesting
@@ -1809,6 +1830,10 @@
This may be called multiple times. It will always return the same. It
can also be interleaved with calls to QCBOREncode_FinishGetSize().
+
+ QCBOREncode_GetErrorState() can be called to get the current
+ error state and abort encoding early as an optimization, but is
+ is never required.
*/
QCBORError QCBOREncode_Finish(QCBOREncodeContext *pCtx, UsefulBufC *pEncodedCBOR);
@@ -1853,7 +1878,7 @@
Normally encoding errors need only be handled at the end of encoding
when QCBOREncode_Finish() is called. This can be called to get the
error result before finish should there be a need to halt encoding
- before QCBOREncode_Finish(). is called.
+ before QCBOREncode_Finish() is called.
*/
static QCBORError QCBOREncode_GetErrorState(QCBOREncodeContext *pCtx);
@@ -2039,7 +2064,7 @@
@retval QCBOR_ERR_UNSUPPORTED Not well-formed, input contains
unsupported CBOR.
- @retval QCBOR_ERR_HIT_END Not well-formed, unexpected ran out
+ @retval QCBOR_ERR_HIT_END Not well-formed, unexpectedly ran out
of bytes.
@retval QCBOR_ERR_BAD_TYPE_7 Not well-formed, bad simple type value.
@@ -2050,6 +2075,9 @@
@retval QCBOR_ERR_EXTRA_BYTES Not well-formed, unprocessed bytes at
the end.
+ @retval QCBOR_ERR_BAD_INT Not well-formed, length of integer is
+ bad.
+
@retval QCBOR_ERR_BAD_OPT_TAG Invalid CBOR, tag on wrong type.
@retval QCBOR_ERR_ARRAY_TOO_LONG Implementation limit, array or map
@@ -2074,6 +2102,10 @@
@retval QCBOR_ERR_NO_STRING_ALLOCATOR Configuration error, encountered
indefinite-length string with no
allocator configured.
+ @retval QCBOR_ERR_NO_MORE_ITEMS No more bytes to decode. The previous
+ item was successfully decoded. This
+ is usually how the non-error end of
+ a CBOR stream / sequence is detected.
@c pDecodedItem is filled in with the value parsed. Generally, the
following data is returned in the structure:
@@ -2194,6 +2226,11 @@
to handle newly defined tags, while using very little memory, in
particular keeping @ref QCBORItem as small as possible.
+ If any error occurs, \c uDataType and \c uLabelType will be set
+ to \ref QCBOR_TYPE_NONE. If there is no need to know the specific
+ error, \ref QCBOR_TYPE_NONE can be checked for and the return value
+ ignored.
+
Errors fall in several categories as noted in list above:
- Not well-formed errors are those where there is something
@@ -2455,6 +2492,18 @@
/**
+ @brief Semi-private method to open a map, array or bstr wrapped CBOR with indefinite length
+
+ @param[in] pCtx The context to add to.
+ @param[in] uMajorType The major CBOR type to close
+
+ Call QCBOREncode_OpenArrayIndefiniteLength() or QCBOREncode_OpenMapIndefiniteLength()
+ instead of this.
+ */
+void QCBOREncode_OpenMapOrArrayIndefiniteLength(QCBOREncodeContext *pCtx, uint8_t uMajorType);
+
+
+/**
@brief Semi-private method to close a map, array or bstr wrapped CBOR
@param[in] pCtx The context to add to.
@@ -2466,6 +2515,17 @@
*/
void QCBOREncode_CloseMapOrArray(QCBOREncodeContext *pCtx, uint8_t uMajorType, UsefulBufC *pWrappedCBOR);
+/**
+ @brief Semi-private method to close a map, array or bstr wrapped CBOR with indefinite length
+
+ @param[in] pCtx The context to add to.
+ @param[in] uMajorType The major CBOR type to close.
+ @param[out] pWrappedCBOR Pointer to @ref UsefulBufC containing wrapped bytes.
+
+ Call QCBOREncode_CloseArrayIndefiniteLength() or QCBOREncode_CloseMapIndefiniteLength()
+ instead of this.
+ */
+void QCBOREncode_CloseMapOrArrayIndefiniteLength(QCBOREncodeContext *pCtx, uint8_t uMajorType, UsefulBufC *pWrappedCBOR);
/**
@brief Semi-private method to add simple types.
@@ -2961,6 +3021,50 @@
QCBOREncode_CloseMapOrArray(pCtx, CBOR_MAJOR_TYPE_MAP, NULL);
}
+static inline void QCBOREncode_OpenArrayIndefiniteLength(QCBOREncodeContext *pCtx)
+{
+ QCBOREncode_OpenMapOrArrayIndefiniteLength(pCtx, CBOR_MAJOR_NONE_TYPE_ARRAY_INDEFINITE_LEN);
+}
+
+static inline void QCBOREncode_OpenArrayIndefiniteLengthInMap(QCBOREncodeContext *pCtx, const char *szLabel)
+{
+ QCBOREncode_AddSZString(pCtx, szLabel);
+ QCBOREncode_OpenArrayIndefiniteLength(pCtx);
+}
+
+static inline void QCBOREncode_OpenArrayIndefiniteLengthInMapN(QCBOREncodeContext *pCtx, int64_t nLabel)
+{
+ QCBOREncode_AddInt64(pCtx, nLabel);
+ QCBOREncode_OpenArrayIndefiniteLength(pCtx);
+}
+
+static inline void QCBOREncode_CloseArrayIndefiniteLength(QCBOREncodeContext *pCtx)
+{
+ QCBOREncode_CloseMapOrArrayIndefiniteLength(pCtx, CBOR_MAJOR_NONE_TYPE_ARRAY_INDEFINITE_LEN, NULL);
+}
+
+
+static inline void QCBOREncode_OpenMapIndefiniteLength(QCBOREncodeContext *pCtx)
+{
+ QCBOREncode_OpenMapOrArrayIndefiniteLength(pCtx, CBOR_MAJOR_NONE_TYPE_MAP_INDEFINITE_LEN);
+}
+
+static inline void QCBOREncode_OpenMapIndefiniteLengthInMap(QCBOREncodeContext *pCtx, const char *szLabel)
+{
+ QCBOREncode_AddSZString(pCtx, szLabel);
+ QCBOREncode_OpenMapIndefiniteLength(pCtx);
+}
+
+static inline void QCBOREncode_OpenMapIndefiniteLengthInMapN(QCBOREncodeContext *pCtx, int64_t nLabel)
+{
+ QCBOREncode_AddInt64(pCtx, nLabel);
+ QCBOREncode_OpenMapIndefiniteLength(pCtx);
+}
+
+static inline void QCBOREncode_CloseMapIndefiniteLength(QCBOREncodeContext *pCtx)
+{
+ QCBOREncode_CloseMapOrArrayIndefiniteLength(pCtx, CBOR_MAJOR_NONE_TYPE_MAP_INDEFINITE_LEN, NULL);
+}
static inline void QCBOREncode_BstrWrap(QCBOREncodeContext *pCtx)
{
@@ -3010,7 +3114,19 @@
static inline QCBORError QCBOREncode_GetErrorState(QCBOREncodeContext *pCtx)
{
- return pCtx->uError;
+ if(UsefulOutBuf_GetError(&(pCtx->OutBuf))) {
+ // Items didn't fit in the buffer.
+ // This check catches this condition for all the appends and inserts
+ // so checks aren't needed when the appends and inserts are performed.
+ // And of course UsefulBuf will never overrun the input buffer given
+ // to it. No complex analysis of the error handling in this file is
+ // needed to know that is true. Just read the UsefulBuf code.
+ pCtx->uError = QCBOR_ERR_BUFFER_TOO_SMALL;
+ // QCBOR_ERR_BUFFER_TOO_SMALL masks other errors, but that is
+ // OK. Once the caller fixes this, they'll be unmasked.
+ }
+
+ return (QCBORError)pCtx->uError;
}
@@ -3024,4 +3140,3 @@
#endif
#endif /* defined(__QCBOR__qcbor__) */
-