C# – Generic BitConverter-like method

I've recently encountered a situation where I need to create a generic method to read a datatype out of a byte array.

I've created the following class:

public class DataStream
{
    public int Offset { get; set; }

    public byte[] Data { get; set; }

    public T Read<T>() where T : struct
    {
        unsafe
        {
            int dataLen = Marshal.SizeOf( typeof( T ) );
            IntPtr dataBlock = Marshal.AllocHGlobal( dataLen );


            Marshal.Copy( Data, Offset, dataBlock, dataLen );


            T type = *( ( T* )dataBlock.ToPointer() );

            Marshal.FreeHGlobal( dataBlock );

            Offset += dataLen;

            return type;
        }
    }
}

Now, de-allocation issues aside, this code doesn't compile with this message:

Cannot take the address of, get the size of, or declare a pointer to a managed type ('T')

Which, seems strange because you should be able to do the above operations based on the where T : struct constraint on the method.

If this code is horribly incorrect, is there any simple way to take a series of bytes and cast them into a 'T' type?

Thanks!