Re: Streaming video over serial?

Michael J. Mahon wrote:
Since the time to transfer a frame is certainly known as soon
as it is captured (even if it is, say, run-length compressed),
the frame rate can be made adaptive to the content. In other
words, if it can run faster, it does.

Yep. In this scenario, since it's doing realtime, maintaining a consistent
frame rate isn't necessary - "as fast as possible" is what's desired. The
closest analogy would be the "remote desktop" applications like VNC, which
attempt to give you a generic mechanism for the local display (and
optionally interactivity) of a remote computer's screen (or app window).

Compression here would also be nice, but
decompression on the Apple might be a killer.

Not if it saves more data transfer time than it costs in processor
time. That's the tradeoff to examine, and it, of course, depends
critically on the actual serial data transfer rate. At 115kbps, it
may be necessary to forego decompression just to maintain the UART
transfer rate. For example, interrupt processing of UART data is
not the way to go at this rate--just simple polling and data storage.

Back-of-the-envelope for 115kbps gives me about a 2Hz update rate for hi-res
graphics without compression. Given that, I'd probably skip lo-res entirely
since that rate is "good enough" for what I'm imagining.

You're right about compression. However, in the particular case I'm thinking
about, the graphics will not be amenable to the sort of decompression the
Apple could do in realtime, at least for intra-frame. Interframe compression
a la MPEG would be feasible to get higher than 2Hz (if less than a full
frame changes, transmit only the rectangle that does), but since it's an
interactive first-person immersive 3D environment the cases where higher
than 2Hz refresh are compelling are when the viewpoint is changing rapidly
(i.e you're trying to naviate), which are the worst-case scenarios for
intra-frame compression.

(That's not to say that either the more complex intermixed lo-res/hi-res or
intra-frame compression wouldn't be useful in more general applications of
this notion, i.e. streaming arbitrary video. I just know enough about the
data in this case to shelve those ideas for now.)

There will be time in the loop to detect a simple protocol escape...

I confess to not knowing enough about serial transmission to know how much
putting a handshake between frames would slow things down, but I have to
assume "not much". If we're transmitting a frame in 0.5 seconds, spending
0.01 second between frames for the client to say "user pressed 'W' key" is
feasible, so doing this within the frame loop is not necessary.

I'm also not sure how the sender knows when when the receiver is done
pulling bits out of a buffer; to reduce perceived latency in this realtime
client/server app, the server should capture the frame to send as close to
when the client is done receiving the previous frame as possible. That's
probably serial communication 101, tho.

When you decide to switch to hi-res, you'll be filling the hi-res
buffer while continuing to display the last full lo-res buffer, I
presume. So if you don't finish transferring the full hi-res
screen, the partial frame transfer would never be visible.

Yep. (Again, it seems that at 115kbps I'd just skip lo-res which simplifies
things)

For highest speed, the protocol will need to be pretty "fragile",
with recovery from a data transmission error essentially awaiting
the next re-syncing event

Agreed. Again, for this scenario, that's fine.

It's looking like the simplest client implementation is basically:
* initialize the UART to 115kbps
* start a loop -
* read from UART until a "start of frame" signature is seen
* start filling the non-visible hires page
* after 8192 bytes, flip hires pages
* jump to start of loop (i.e. wait for a signature)

The next thing to implement would be a simple 1-byte sync signal every 256
bytes; if not seen, assume there was a glitch somewhere and abort the
display of this frame, waiting for the next start-of-frame signature.

I've never done serial programming on either side, but I'm assuming the
initialization is trivial and you've provided the guts of the loop already.
(Thanks!)

The hard part now seems like it's actually on the sending side, where we
need to tackle algorithms for generating decent color hi-res screens. I'd
probably start by assuming it's a 140x192 with a fixed 6 color palette, do
an error-diffusion dither, and ignore the artifacts from trying to put blue
and green within (etc) a byte. Which would make Rich happy, I'm assuming.

Joshua


.

Relevant Pages

  • Re: Mini-DV vs Analog to the Desktop - Question...
    ... and Digital 8 camcorders record 29.97 frames per second, just like analog ... there is absolutely no difference in frame rate between analog ... You claim DV is better than MPEG2 ... By it's nature, it entails compression of the image at, ...
    (rec.video.desktop)
  • Re: video capture
    ... DVDs are substantially more compressed than ... but DV is a different style of compression than what is ... that each frame is a complete image. ... Capturing baseband video uncompressed requires special ...
    (rec.video.production)
  • Re: Streaming video over serial?
    ... In this scenario, since it's doing realtime, maintaining a consistent frame rate isn't necessary - "as fast as possible" is what's desired. ... Back-of-the-envelope for 115kbps gives me about a 2Hz update rate for hi-res graphics without compression. ... about, the graphics will not be amenable to the sort of decompression the Apple could do in realtime, at least for intra-frame. ... The hard part now seems like it's actually on the sending side, where we need to tackle algorithms for generating decent color hi-res screens. ...
    (comp.sys.apple2.programmer)
  • Re: DV: digital vs. analog dubs
    ... Look, you can claim all you like the data dropouts don't happen on DV, ... where the DV deck can't mask it and drops in a frame of silence ... Decks may do this kind of correction, ... In interframe compression one of the main ways of saving space is to reduce ...
    (rec.video.production)
  • Re: help analyzing low system(with sar/vmstat/u386mon/sarcheck data)
    ... Device MAC address in use Factory MAC Address ... not pass the Frame Check Sequence ... first transmission attempt ... the system 450MB of buffer cache. ...
    (comp.unix.sco.misc)
Loading