Parsed byte array turns into garbage after being parsed into a struct

I am receiving data packets over UDP that I parse into structs and in a specific situation my data gets all messed up and I can't figure out what the problem might be. Let me give you an overview of what I am doing.

1. UDP Receiver receives the data packet and passes it to a callback - size of the byte array at this stage is 1307 bytes which is the expected size for the packet
2. The byte array is marshalled (is that the right term?) into a struct - size of the struct is now 1484 bytes (not sure if that is relevant)

In this example the data is send from the Formula 1 2020 game and the curious thing is that the first entry in the m_carTelemetryData array (see below) is always fine, the data is all sound, however every entry of the 22 after the first one is totally messed up with either 0 values, null values or completely outlandish values for all the different fields in the struct.

I tried several things already to pinpoint the issue but have now reached the end of my knowledge about the things I am dealing with here. My best guess is that something is going wrong when converting the data into a struct or something else is going on that causes a misalignment (?) of the data.

What I tried so far

• Changed my code from "magic marshalling" to manual reading the data byte-by-byte using BinaryReader - no luck
• Manually checked the data using BitConverter.ToFoo(bytes, offset) - no luck
• Yolo changed the Pack attribute assuming that's where I got wrong - no luck
• Double-checked the documentation to make sure I got the data types right - I am fairly confident that I "translated" them correctly
• Banged my head against a wall - still no luck

My questions:

• Is there something obvious wrong with my code that I am simply not seeing?
• Side question: am I wrong in my assumption that the size of the struct should match the suze of the byte array it has been created from?

Here is the code for reference (if anything helpful is missing, please let me know):

Packet Header

[StructLayout(LayoutKind.Sequential), Pack = 1]
struct PacketHeader2020 {
    public ushort m_packetFormat;
    public byte m_gameMajorVersion;
    public byte m_gameMinorVersion;
    public byte m_packetVersion;
    public byte m_packetId;
    public ulong m_sessionUUID;
    public float m_sessionTime;
    public uint m_frameIdentifier;
    public byte m_playerCarIndex;
    public byte m_secondaryPlayerCarIndex;
}

Packet

public struct CarTelemetryPacket2020
{
    public PacketHeader2020 m_header;
    
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 22)]
    public CarTelemetryData2020[] m_carTelemetryData;

    public ButtonFlag m_buttonStatus;
    public byte m_mfdPanelIndex;
    public byte  m_mfdPanelIndexSecondaryPlayer;
    public byte m_suggestedGear;
};

CarTelemetryData2020

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct CarTelemetryData2020
{
    public ushort m_speed;
    public float m_throttle;
    public float m_steer;
    public float m_brake;
    public byte m_clutch;
    public sbyte m_gear;
    public ushort m_engineRPM;
    public byte m_drs;
    public byte m_revLightsPercent;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    public ushort[] m_brakesTemperature;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    public ushort[] m_tyresSurfaceTemperature;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    public ushort[] m_tyresInnerTemperature;
    public ushort m_engineTemperature;
    
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    public float[] m_tyresPressure;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    public SurfaceType[] m_surfaceType;
}

byte[] -> struct

public static T ByteArrayToStructure<T>(byte[] bytes) where T : struct
{
  var handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
  try
  {
    return (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
  }
  finally
  {
    handle.Free();
  }
}