#2828(avformat:new): HLS segment MPEGTS continuity counter is being incorrectly set to 0 on each segment

Discussion:

FFmpeg

2014-04-24 15:51:56 UTC

FFmpeg

2014-07-28 16:08:44 UTC

#2828: HLS segment MPEGTS continuity counter is being incorrectly set to 0 on each
segment
-------------------------------------+------------------------------------
Reporter: dlevinson5 | Owner:
Type: defect | Status: new
Priority: normal | Component: avformat
Version: unspecified | Resolution:
Keywords: hls mpegts | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+------------------------------------

Comment (by trnsz):

I am having the same problem. I am also not using the ffmpeg tool, but
the libraries directly, but the problem can easily be made to show itself
using the tool as well:

Here are unredacted commands and outputs.

(Output too big to post here when trying to submit, so here is pastebin):
$ ffmpeg -i http://download.wavetlan.com/SVV/Media/HTTP/MP4/ConvertedFiles
/Media-Convert/Media-
Convert_test1_36s_AVC_VBR_521kbps_320x240_25fps_AACLCv4_VBR_96kbps_Stereo_44100Hz.mp4
-c copy -map 0:v -map 0:a -bsf h264_mp4toannexb -flags -global_header
-segment_list_flags +live-cache -segment_time 2 -segment_list_size 30
-segment_wrap 60 -segment_time_delta 0.08 -segment_list ./test.m3u8 -f
ssegment "./test%05d.ts" -report

http://pastebin.com/aScZgCnN

Now let's read it back:

(Output too big to post here when trying to submit, so here is pastebin,
again):

$ ffmpeg -debug 1 -i test.m3u8 -f null /dev/null -report

http://pastebin.com/kyd6k6rt

As you can see, at the start of each segment, there are failed continuity
checks as the timestamp is incorrectly reset at the start of each new
segment:

(Excerpt from above)

[hls,applehttp @ 0x1b54a20] HLS request for url 'test00003.ts', offset 0,
playlist 0
[mpegts @ 0x1b573e0] Continuity check failed for pid 17 expected 5 got 0
[mpegts @ 0x1b573e0] Continuity check failed for pid 0 expected 8 got 0
[mpegts @ 0x1b573e0] Continuity check failed for pid 4096 expected 8 got
0
[mpegts @ 0x1b573e0] Continuity check failed for pid 256 expected 7 got 0

This will cause the output to fail Apple validation and is breaking
specification, so technically the output is corrupted. While the output
does play in ffmpeg, VLC, ffplay, etc, but gives warnings similar to above
in all cases.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/2828#comment:16>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

FFmpeg

2014-07-28 16:38:35 UTC

Permalink

#2828: HLS segment MPEGTS continuity counter is being incorrectly set to 0 on each
segment
-------------------------------------+------------------------------------
Reporter: dlevinson5 | Owner:
Type: defect | Status: new
Priority: normal | Component: avformat
Version: unspecified | Resolution:
Keywords: hls mpegts | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+------------------------------------

Comment (by trnsz):

Interesting, according to a W3C document at https://dvcs.w3.org/hg/html-
media/raw-file/d5956e93b991/media-source/media-source.html#mpeg2ts-
discontinuities it is stated that clients should handle TS
discontinuities:

{{{• When a discontinuity is detected, MPEG2TS_timestampOffset must be
adjusted to make the timestamps after the discontinuity appear to come
immediately after the timestamps before the discontinuity.}}}

However, the Apple documentation for the validation tool at
https://developer.apple.com/library/ios/technotes/tn2235/_index.html#//apple_ref/doc/uid/DTS40010221
specifies:

{{{
WARNING: stream discontinuity detected

You must use the EXT-X-DISCONTINUITY tag where an encoding discontinuity
exists between the
media file that follows it and the one that preceded it. See the IETF
Internet Draft of the
HTTP Live Streaming Protocol Specification for more information.

If you intend to play a new stream at a given point in your presentation,
you must use the
EXT-X-DISCONTINUITY tag.
}}}

So a workaround that would be a quick hackish would might be to change the
M3U8 output to include the EXT-X-DISCONTINUITY tag between all segments
and should result in ffmpeg output passing the Apple validation tests.

The proper fix is still maintaining the timestamp across segments.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/2828#comment:17>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

FFmpeg

2014-07-28 17:19:56 UTC

Permalink

I have extracted your changes from your original mpegtsenc.c...
[snip]
On the other hand, if ff_get_continuity_counter() and

ff_set_continuity_counter() actually existed, that might be a cleaner
solution for everyone...

dicroce, if you are still watching and care to post the patch here, it
would save me some time. If not I'll can work up something quickly
myself. I'm hoping that a "real" ffmpeg developer would look into this
first, because my unfamiliarity with the ffmpeg internals means any patch
I submit would be less likely to be accepted than one by another developer
more versed with the code.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/2828#comment:18>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

FFmpeg

2014-08-23 14:55:17 UTC

Permalink

FFmpeg

2014-08-23 15:24:06 UTC

Permalink

Post by FFmpeg
Because counters are kept in global statics

This means that the patch unfortunately cannot be applied;(

Can you explain how I can reproduce the issue with {{{testsrc}}} input and
without using external libraries?

--
Ticket URL: <https://trac.ffmpeg.org/ticket/2828#comment:20>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

FFmpeg

2014-08-23 17:10:42 UTC

Permalink

#2828: HLS segment MPEGTS continuity counter is being incorrectly set to 0 on each
segment
-------------------------------------+------------------------------------
Reporter: dlevinson5 | Owner:
Type: defect | Status: new
Priority: normal | Component: avformat
Version: unspecified | Resolution:
Keywords: hls mpegts | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
-------------------------------------+------------------------------------

Comment (by lancey):

Well, as I said these counters need to be moved to some structure related
to the output, which is kept across segments, so there are copies per
output.

You can reproduce the issue with the following command:

ffmpeg -i test.ts -codec copy -flags global_header -map 0 -f segment
-segment_time 10 -segment_list_size 6 -segment_wrap 20 -segment_list
playlist.m3u8 pl%03d.ts

This will output one ts file per 10 sec. Inspecting each ts file you'll
see CC counters for each PID start from 0, instead of the appropriate
number which the last PID packet had within the previous segment. I've
personally not encountered a scenario where this breaks playout on any
device/player, but some people do. Also, playling the result with VLC
gives warnings as well (but no playout disturbances occur). Per HLS specs,
counters should indeed be kept across segments, so we should fix this for
sure.

Also, it would be even better if we keep the incoming CCs in case we are
copying the stream.

If you need example files, both input and output, let me know.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/2828#comment:21>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

FFmpeg

2014-08-23 18:47:49 UTC

Permalink

FFmpeg

2014-08-23 18:50:44 UTC

Permalink

FFmpeg

2014-08-23 19:37:18 UTC

Permalink

FFmpeg

2014-08-23 21:48:28 UTC

Permalink