Hardstack project preliminary API specifications and goals.

stman

Hello,

Hardstack is the name of a development project I have initiated in march 2024, that is aiming at creating a standalone module, all implemented in cabled logic (No CPU, no CPU code, no OS) either into an FPGA or using discrete logic gates (74xxx family), that embed :

A self-made PHY driver for 10BASET ethernet (10 Mbps) made out of analog and discrete logic components.
A self made simplified Ethernet controller, made out of FSMs in cabled logic.
A self-made simplified elementary TCP/IP stack having all what is necessary to handle TCP sockets and UDP sockets, all designed in cabled logic too with several FSMs.

Hardstack developement prototype PCB combined with a low cost ($40) Spartan 6 Dev Board :

Hardstack shall be seen as a standalone TCP/IP stack module, provenly safe and secure, in cabled logic. There is no CPU, no CPU code, no OS involved in its operations, it is fully designed in cabled logic.

The primary goal of Hardstack is to be used as a “component module” in order to create super safe servers or P2P nodes, in a way those hosts or node cannot be hacked or pwnd through the many backdoors and exploitable security found into usual PC hardware and BIOS/Operating systems and kernels regarding the TCP/IP stack.
A secondary goal of Hardstack is to get rid of all known side channels and hidden channels of all sorts that can be constructed over a TCP/IP stack, from the backdoored ethernet and PHY controlers, up to the high parts of TCP/IP stack implemented in complex software into Kernels and/or OS. Eliminating all side and hidden channels through a known deterministic and fixed simplified implementation of the whole TCP/IP stack from the PHY controller up to UDP and TCP sockets management is a geat achievement in terms of security, particularily regarding anonymity issues, as we can finaly garantee that using such module connected to a standard PC allows to bypass its usual PHY+Ethernet controler and TCP/IP software stack, an use Hardstack instead, garanteeing, with predictability, that none of the well known and highly documented side / hidden channels over TCP/IP can be used by any malware or backdoors into an OS in order to “tag TCP/IP traffic” (Tag all the datagram emitted) covertly with some unique identifiers allowing accurate identification of a machine over the internet, by an attacker like NSA that has the means and computing power to scan the whole internet traffic and to detect those covert tags added into side/hidden channels over TCP/IP.

In this regard, using Hardstack module is a “must have” complement to the famous Tor router, or to any VPN configuration. It solves by design all the many accurate identification technics based on tagging covertly all datagrams with unique identifiers into side / hidden channels over TCP/IP.

The following article analyse and details all those well known side / hidden channels over TCP/IP, for your information in case you would not be familiar with them : (In French, but it can be easily translated into english, and a lot of references and sources listed at the end of the paper are english publications) : https://wiki.deimos.fr/images/a/a8/Canaux_cach%C3%A9s_(ou_furtifs).pdf

The third goal is a segmented pipeline approach of networking through the use of a first module, hardstack (That handles the whole TCP/IP stack up to TCP connexion, TCP data exchange or UDP datagram exchange), that can be connected to other modules decoding highler levels application specific protocols and performing a first check pass, and then such secundary “proxy module” connected to the final PC serveur motherboard.

In such “pipelined” hardware configuration, it is fully different and autonomous hardware modules that are connected in daisy chain starting from Hardstack up to the PC motherboard being used as a server.

See examples of Hardstack setups belows :

The fourth goal of Hardstack is to separate “data exchange API” (both hardware and software API) from the “hardstack configuration and socket connexion and configuration parameters API”. In other word, we can build safer servers because they actually don’t have any control over socket connexions and their parameters is the server only connects to Hardstack with the Data Exchange API, and not with the “Harstack & Socket configuration and management API”.

In most cases, when building servers, the necessary Hardstack configuration is “fixed” and doesn’t actually need any dynamic configuration from the server : You just need to configure Hardstack module to “listen” to given port(s) and you’re done.

Preventing the server motherboard PC from controlling Hardstack sockets, their IP and Mac parameters, their connexion status in the case of TCP connection, means that even if your server is full of malware or govware or backdoors, they won’t be able to establish dedicated communications to a C&C server, because the server motherboard is NOT connected to the secund API of Hardstack used to configure its operations and its sockets parameters and connexion status.

The fith core functionality is that I plan a mode where current IP/mac addresses are copied in read-only to the “data exchange API” when receiving data, or when sending some, but they cannot be changed through this interface (This can be usefull when using UDP socket in packet operations, where the node/server needs to know, for each UDP datagram received, the originating IP Address in order to distinguish several remote hosts sending UDP datagrams data to such UDP socket slot. But I also plan a mode where the real originating IP is not transfered over the “data exchance API”, but replaced by a random identifer generated by Hardstack, allowing to still distinguish several different originating IP addresses, while actually ignoring their true values, see that as a random alias of originating IP addresses. This functionality is usefull to enhance anonymity and security of servers/nodes : They ignore the actual peer they are exchanging data with, only Hardstacks knows them for real. This is another level of security for anonymity matters, preventing any backdoor or malware or govware in the server from exfiltrating any TCP/IP related info for the remote originating addresses it is communicating with through any known mean (side channels, whatever), simply because they ignore them. This is important for servers where we want to ensure anonymity is really protected with several technics the best we can.

The goal of this thread is to present the two proposed API (Both Hardware and software for each of them) for the “Data Exchange API” used by the server to exchange data over connected sockets, and the “Hardstack and socket configuration, socket connexion and parameters API”, and have some feed back over some possible improvement.

This thread will also present the internal architecture block diagram of Hardstack, that is very different from usual Ethernet Controllers ICs + PHY ICs, with software TCP/IP stack implemented in an OS / RTOS : As Hardstack integrate the whole stack in a standalone module, and has it is voluntarily limited to the strict necessary implementation to enable UDP or TCP communications, the internal bloc diagram architecture has been optimized with such goal in mind : It is “socket slot” oriented. Each socket slot handle a single socket, either TCP or UDP. Each slot has its own independent TCP/IP parameters (Hardstack host Mac address, Hardstack Host socket source static IP address, destination IP address, router’s IP address, mode of operation (Outgoing connexions only, incoming connexions only, or both), for each slots.

To be continued with a new post regarding the two API interfaces, both at hardware level, and then “registers/custom protocol levels” for the “Hardstack configuration and Socket operations interface”, and for the “socket data exchange interface”, that form the two direct API from which the Hardstack module can be fully controlled, keeping in mind that in most use cases, to build servers or P2P nodes with Hardstack, only the “data exchance API” needs to be connected to the node/server motherboard, the other API can be connected to a standalone microcontroler or EPROM to setup Hardstack’s configuration and operation persistently, without involving the node/server motherboard.

stman

HARDSTACK APIs PROTOCOL

Hardstack being a cabled logic engine made out of multiple FSMs and mainly coded in VHDL, the communication protocol over the two asynchronous serial ports, both for the Data API and the Configuration API between Hardstack module and a PC Server must be as simple and secure as possible (Side channel safe), so that the implementation of these protocols CODECs stays as simple as possible through FSMs in VHDL, but also in the language of your choice on the secure PC server. Simplicity is key for security, and state-of-the-art secure implementation in general.

GENERAL REMARKS ON THE PROTOCOL

For both API over the two asynchronous serial ports, there is no master of slave notion for the communication protocol. The protocol is fully balanced. Both the PC Server and Hardstack can send a request/command/data to the other side when something is needed, then the other side has to acknowledge it. We don’t any Frame transmission bit errors, but we provide with this a basic error detection and retransmission mechanism in case of transmission errors occurs on the serial lines, in can always happen from time to time as we use high speed clocks. It is recommended to use a short and well shielded cable. Any side who sends a given command cannot send another same command type until the other side acknowledged it first, this means that in each transmission direction, there is a maximum of one pending command/request sent “per command/request type codes and per socket slot”, always. Acknowledgment frames don’t need to be acknowledged themselves.

For this documentation of the protocol format used for the two serial ports used respecively for the Config API and the Data API of Hardstack, each bullet list item describes a given field meaning for a given message frame, its size in bytes (Most of the time a fixed value), and its encoding informations (Sometimes such field has a bitmapped content). All fields on more than one bytes are always Big Endian encoded, because Little Endian suchs for debugging.

Frames incremental counter numbers are local to each side of the serial link. Each starting from zero for each side, and looping to zero when reaching 255 value.

DATA API PROTOCOL

Hardstack -> PC Frames :

Acknowledgement for PC -> Hardstack frames :

These frames are sent immediatly by Hardstack to acknowledge each acknowledgeable frame received from the PC. Through it, Hardstack can request to the PC to resent the last frame sent, if it contains a CRC 32 check error.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “PC Frames Acknowledgment” <0×03> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Flags (Bitmap : Bit 1 : 0 = Received Frame CRC OK, 1 = Received Frame CRC ERROR Retransmit the Erroneous Frame ) : 1 Byte
PC Frame Number to Acknowledge : 1 Byte
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

Hardstack Reset or Power-Up :

This frame is sent by Hardstack everytime it performs a Reset or a Power-Up. This frame also means that frames counters on both sides shall be reinitialized to zero after the frame is received. This frame is not acknowledgeable (It has no frame counter field). This frame also informs the PC server of some key parameters necessary to implement the protocol properly, including some necessary informations regarding the Socket Slots configuration for each socket slot.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “Hardstack Reset or Power-Up” <0×07> : 1 Byte
Protocol Version - Current version is <0×00> : 1 Byte
Number of Socket Slots (Referred to later on in this document as NSS) installed and operating (8 Sockets Slots available means a value of <0×08>) : 1 Byte
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

Ethernet Link Status :

This frame is sent by Hardstack everytime its internal state of the Ethernet Physical Link changes. After a Reset or a Power-Up, the internal Ethernet Physical Link status is always set to “NO LINK”. Hardstack informs the PC server of its evolution with this frame, as it changes over time.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “Ethernet Link Status” <0×09> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Link Status, encoded in two bits (Encoding : <0×00> = NO LINK, <0×01> = NLP TX, <0×02> = NLP RX, <0×03> = LINK OK) : 1 Byte
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

TCP Socket Connexion :

This frame is used to signal the PC that a given socket is now connected. From the reception of this frame by the PC, Hardstack will begin transferring data to the PC for this socket. Similarly, the PC can start sending data for this socket to Hardstack after he receives this frame.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “TCP Socket Connexion” <0×01> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Socket slot number (Valid value range : 0 to NSS-1) : 1 Byte
Flags (Bitmap : Bit 1 : 0 = Remote Host IP Address Mode, 1 = Remote Host Anonymous ID Mode ) : 1 Byte
Remote Host IP Address (IP Address or Anonymous ID) : 4 Byte
Remote Host IP Port (IP Port or Anonymous ID) : 2 Bytes
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

TCP Socket Disconnection :

This frame is used to signal the PC that a given socket is now disconnected. No more socket data will be transmitted to the PC for this socket after this frame is sent to the PC. The PC shall not send more socket data frames to the given socket after this frame is received, they would simply be ignored if he dares to.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “TCP Socket Disconnexion” <0×04> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Socket slot number (Valid value range : 0 to NSS-1) : 1 Byte
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

TCP Socket Data :

This frame is used by Hardstack to transfer data received from the Ethernet Link on a connected TCP socket. Hardstack will never send such frame if the socket is disconnected. Hardstack will always transfer all its pending received data for a socket to the PC with this frame, before sending a disconnect frame if the sockets gets disconnected. This way, the PC has the garantee to always received all the received data for a given socket, even when the socket gets disconnected.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “TCP Socket Data” <0×05> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Socket slot number (Valid value range : 0 to NSS-1) : 1 Byte
Flags (Bitmap : Bit 0 : 0 = No more data to come after these ones, 1 = More data to come) : 1 Byte
Data size N (Unsigned and coded with 15 bits, the most significant bit must be zero) : 2 Bytes
Data : N Bytes
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

UDP Socket Data :

This frame is used by Hardstack to transfer data received from the Ethernet Link on a UDP socket slot to the server. As UDP is connexion-less, and as a UDP socket slot can be configured to accept UDP datagrams from only one or from several different remote hosts, each UDP datagram received by Hardstack is always associated to a remote host IP and port. When the anonymous mode is used for a given Socket Slot, the remote host IP and Port are replaced by an anonymous identifier coded on 6 bytes and guaranteed constant over time.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “UDP Socket Data” <0×0A> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Socket slot number (Valid value range : 0 to NSS-1) : 1 Byte
Flags (Bitmap : Bit 0 : 0 = No more data to come after these ones, 1 = More data to come -
Bit 1 : 0 = Remote Host IP Address Mode, 1 = Remote Host Anonymous ID Mode ) : 1 Byte
Remote Host IP Address (IP Address or Anonymous ID) : 4 Byte
Remote Host IP Port (IP Port or Anonymous ID) : 2 Bytes
Data size N (Unsigned and coded with 15 bits, the most significant bit must be zero) : 2 Bytes
Data : N Bytes
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

PC -> Hardstack Frames :

Acknowledgement for Hardstack -> PC frames :

These frames are sent immediately by the PC to acknowledge each acknowledgeable frame received from Hardstack. Through it, the PC can request to Hardstack to resent the last frame sent, if it contains a CRC 32 check error.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “Hardstack Frames Acknowledgment” <0×02> : 1 Byte
Incremental PC Frame Number Counter : 1 Byte
Flags (Bitmap : Bit 1 : 0 = Received Frame CRC OK, 1 = Received Frame CRC ERROR Retransmit the Erroneous Frame ) : 1 Byte
Hardstack Frame Number to Acknowledge : 1 Byte
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

PC Reset or Power-Up :

This frame is sent by the PC server everytime it performs a Reset or a Power-Up of its server application (Meaning when it starts opening and handling the serial port with this protocol. This frame also means that frames counters on both sides shall be reinitialized to zero after the frame is received. This frame is not acknowledgeable (It has no frame counter field).

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “PC Reset or Power-Up” <0×08> : 1 Byte
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

TCP Socket Data :

This frame is used by the PC to transfer data on a given TCP socket, if it is connected. If the socket is not connected, this frame will be ignored by Hardstack. If the amount of data to transfer into the socket is less or equal to the maximum size transferable with such frame, you shall set the “No more data to come after these ones” Bit 0 to zero in the Flags field : This guarantees that these data will be sent immediately by Hardstack into the socket to the remote peer. If amount of data to transfer into the socket is greater than the maximum size transferable with such frame, you shall set the “More data to come” Bit 0 to one in the Flags field. This bit controls the internal mechanism in Hardstack that wait to fill a full datagram MTU before sending it or not.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “TCP Socket Data” <0×06> : 1 Byte
Incremental PC Frame Number Counter : 1 Byte
Socket slot number (Valid value range : 0 to NSS-1) : 1 Byte
Flags (Bitmap : Bit 0 : 0 = No more data to come after these ones, 1 = More data to come) : 1 Byte
Data size N (Unsigned and coded with 15 bits, the most significant bit must be zero) : 2 Bytes
Data : N Bytes
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte

UDP Socket Data :

This frame is used by the server to transfer data to an UDP socket slot on Hardstack. As UDP is connexion-less, and as a UDP socket slot can be configured to accept UDP datagrams from only one or from several different remote hosts, each UDP data sent to Hardstack is always associated to a remote host IP and port. When the anonymous mode is used for a given Socket Slot, the remote host IP and Port are replaced by an anonymous identifier guaranteed constant over time.

Start of Frame - STX : <0×02> : 1 Byte
Frame content size in bytes, excluding SOF and EOF bytes : 2 Bytes
Frame message type code for “UDP Socket Data” <0×0B> : 1 Byte
Incremental Hardstack Frame Number Counter : 1 Byte
Socket slot number (Valid value range : 0 to NSS-1) : 1 Byte
Flags (Bitmap : Bit 0 : 0 = No more data to come after these ones, 1 = More data to come -
Bit 1 : 0 = Remote Host IP Address Mode, 1 = Remote Host Anonymous ID Mode ) : 1 Byte
Remote Host IP Address (IP Address or Anonymous ID) : 4 Byte
Remote Host IP Port (IP Port or Anonymous ID) : 2 Bytes
Data size N (Unsigned and coded with 15 bits, the most significant bit must be zero) : 2 Bytes
Data : N Bytes
CRC 32 of all frame bytes, excluding SOF, CRC 32, and EOF bytes in the frame : 4 Bytes
End of Frame - ETX : <0×03> : 1 Byte