F1 begins urgent investigation into faults which blighted F1 TV launch

2018 F1 season

Posted on

| Written by and

Formula One Management has begun an urgent investigation into the problems which plagued the launch of its F1 TV service last weekend, RaceFans understands.

Fans who tried to watch the Spanish Grand Prix using the new live streaming service experienced buffering problems (pictured) and other faults which rendered the stream unwatchable for some. F1 says it has already offered refunds to customers who were affected.

F1 TV presents a significant technical challenge. It involves broadcasting two dozen live streams in 1080p HD quality from a range of different international locations, while also ensuring the footage can only be accessed in certain regions.

FOM has enlisted the services of several different suppliers to deliver F1 TV. The problems it has experienced are understood to involve making those technologies work together seamlessly to deliver live playback.

F1’s punishing schedule offers little time to implement fixes. The next live session begins in nine days in Monaco. After that the series moves on to Canada, then returns to Europe for five races within six weeks.

According to FOM the problems it encountered last weekend “only affect customers during live playback.”

“All sessions from this weekend’s Spanish Grand Prix will be available to enjoy on-demand including all onboard cameras and full team radio,” it added. “Full replays are available to F1 TV Pro customers, or F1 TV Access customers in selected territories.”

F1 TV has already suffered delays. Its planned launch for the beginning of the 2018 F1 season was postponed by four races when FOM realised it wouldn’t be ready for the opening four ‘fly-away’ rounds.

Fans reveal F1 TV faults

Image: KO Racing via YouTube

Advert | Become a RaceFans supporter and go ad-free

2018 F1 season

Browse all 2018 F1 season articles

67 comments on “F1 begins urgent investigation into faults which blighted F1 TV launch”

  1. It was a difficult problem 10 years ago but not anymore, there are many people who know how to do it and did it elsewhere. There really isn’t an excuse for this.

    1. @ivan-vinitskyy As someone who doesn’t follow non-motorsports that closely I’m curious to know how other sports’ offerings compare. Are there others trying to broadcast this many live feeds simultaneously under similar conditions?

      Obviously you’ve got the huge media companies like Netflix but there’s a reason why FOM are trying to do it all themselves and I’m wondering if any other sports have done the same successfully.

      1. Matteo (@m-bagattini)
        15th May 2018, 12:38

        @keithcollantine maybe @ivan-vinitskyy is referring to big competitors like Netflix or Youtube. The question here is why they didn’t partner with one of these companies since it’s their job and they proved they can do it quite well.

        For example, for many years Ferrari struggled with their live streaming of the F1 car unveil. Their job is to make cars, not stream media content. Look at what SpaceX does with their launches: they let YouTube care about the broadcast.

        You’re saying there’s a reason why FOM are trying to do it all themselves, can I ask you if you know this reason?

        1. Money and ownership.

          I.e. if YouTube was to stream they take he larger portion of ad revenue and Liberty have no control over ads etc.

          Whilst it is painful at the start the investment Liberty are making in F1TV is a long play. I would imagine it has a 3-4 year ROI involved and everything at the moment is a a strategic loss.

          To be honest, I respect Liberty for wanting so much functionality at the start. They could have easily half assed it and repurposed the sky stream for example. 1 feed is easier than 12+ other data sources. It shows both their aspirations and expectations.

        2. I think that on the one hand they are competitors with the likes of Netflix, Disney, Amazon and youtube @m-bagattini, and they do not want to be reliant on a single party. On the other hand it’s quite something different to have a real international platform than what Netflix/Amazon etc do, since you always sign up to the LOCAL offering, which can hugely vary per territory.

          To latch on to your argument about Ferrari and the live stream, you actually make a great point. But you failed to understand that for Liberty this IS what they do – they provide AND offer content to their viewers. FOM is part of that content, and maybe it is now building a worldwide sports network that can potentially compete with the likes of Amazon or Disney (since Netflix doesn’t offer any live sports as far as I am aware)

        3. Oh, we can’t compare F1 TV to Netflix and its peers. Netflix and co. have the benefit that their offering is not live content generated mere seconds earlier, so they are able to use their CDN partners to cache it geographically.

          F1 TV is pushing live streams from 20 cars + the Sky stream in near realtime, and it has to send it globally. The very breadth of F1’s fanbase is what proves the challenge (unlike, say, NBA TV which is mostly in the US, and presumably a comparatively smaller number of US expats and other NBA fans outside the US, if at all).

          Yes, YouTube does host some live streams, and yes, Google’s breadth might have helped alleviate these initial problems. However, I fully support FOM’s approach of rolling their own content platform, for the same reasons that @captainpie has mentioned – money and ownership, as well as one more I’d add – branding.

          Liberty Media is a media company, and for them to get hosted on another large media platform would not help their brand. Using Tata/TCS is different, as they are seen as an IT company, and not a media platform. It’s one thing to have a line at the bottom that says “Powered by TCS”, it looks quite different for that line to read “Powered by YouTube technology”.

          I sympathize with Liberty for the technical issues they face, but will blame their poor decision making in rolling this out in such a manner. I said this before – it’s better to have a delayed service, than one that launches at the start of the season but is riddled with problems. Sadly, Liberty have gone the route of combining the worst of both approaches.

          1. This. Couldn’t have said it better myself.

      2. @keithcollantine, @ivan-vinitskyy, I doubt there really is any product that is comparable and does work.

        Which sport has a different venue every week to three weeks, is watched globally and has a choice of several packages depending on local territory of the watcher?

        For most sports, it would be either from regular locations (football stadiums within one country or select ones within the EU for the Champions league – who by the way does not have a comparable international product) or there are single big events that are run from a few locations that changes every few years (Olympics, Football world cup, other sports world championships).
        Something like Tennis or skying or biathlon might offer comparable circumstances, but none of them have anything like this available for their world championships.
        Probably the closest would be packages that are offered in some regions for football, cricket, basketball etc. But they don’t have as many factors combined either.

        I think it is well possible that this can be something that Liberty is developing to be then able to offer to other sports as well, i.e. an investment for a very interesting and long future.

        1. That final line right there. This is Liberty Media’s investment in future distribution. Remember this is what Liberty’s core business is, MEDIA distribution. They own plenty of outlets for this and have bought more than a few other online networks and video producing companies. Better distribution of this information and media is their stated goal.

        2. Rallying and cycling might be somewhat in the same category, and from what I’ve heard, WRC+ All Live coverage works surprisingly well. Then again, the numbers are completely different when comparing WRC and F1.

          1. Yeah, I am pretty sure the scale of WRC viewership (and probably the rights deals are not as mixed a bag in as many territories either both because of the scale/amount of money involved and well, BE having done the F1 deals) @kaiie

        3. @bascb @m-bagattini @keithcollantine @ivan-vinitskyy I can tell it remains extremely difficult even today. Few examples can be found recently, from FIFA world cup to Baumgartner jumping from space, with even eSports finales (League of Legends on twitch.tv for example).
          Most of the time there are only a few players able to handle that. Namely Google, or decent infrastructures like Akamai. Problem is, Tata is not Akamai, the FIA underestimated the task and certainly shortened the development roadmap.

        4. Evans Armitage
          15th May 2018, 18:12

          WTA works. They don’t have the equivalent for multiple onboards though.
          In motorsports, WEC and Formula E work fine. The onboards of Porsche cars worked fine when they were in WEC.
          They should probably work with the teams for this to some extent to help them reduce the load. The teams also benefit after all and each carry some infrastructure around for pushing gigs of data in real time.
          I wonder if their IT architects considered companies that already have the infrastructure with autoscaling built in like AWS https://aws.amazon.com/answers/media-entertainment/live-streaming/

      3. Umar A (@umartajuddin)
        15th May 2018, 14:47

        @keithcollantine NBA TV is an excellent comparison, and probably the leader in this space. There are a slew of complex TV contracts across north america that dictate how and where games can be shown, but NBA TV navigates this really well and offers packages for each region that dont violate those. It also has shot charts, box scores and live analysis the stream is HD, and I dont recall too many big glitches in the last few years. If I remember correctly, its been around since 2008 so they’ve had time to get the technology right.
        Though to be fair, the NBA doesnt reach 400+ million viewers, its only about half of that.

      4. Personal opinion, as someone who actually mucks about with large, highly available websites, streaming of the magnitude Liberty is trying to accomplish is not easy. Getting a few thousand bytes (a typical webpage) from point a to point b in a timely fashion isn’t difficult these days (although there’s an awful lot of content caching involved to make it feel snappier).

        Getting 24 or 30 frames-per-second of HD footage across the globe in a timely fashion is not easy, and frankly, I’d have been astonished if it hadn’t collapsed. World Cup feeds, Olympic feeds, other major sporting events– they’ve all stumbled out of the gate, because they’re a unique problem each time. The good news is, it’s not a difficult problem to solve, it just takes time. And a lot of money.

        1. It’s not THAT easy. Single sign-on technology, DRMs, adaptive streaming for various devices (HLS, DASH, MSS) is very tricky to get working seamlessly. And it’s the law of the diminishing returns – make it work for 80% of the users is relatively easy, taking care of the 20% left eats way more money and time. Also, app coding for performance is tricky and requires high level developers.

          1. Also when you consider the physical infrastructure investment and logistics of setting this up once a year at each location. It’s either the same gear that you move everywhere around the globe and need to feed this into a streaming media services using whatever network you can build to each location, or you contract local infrastructure providers to do this, which means you don’t necessarily follow the same standard in each corner of the globe.

      5. @keithcollantine NFL does a great job, letting you follow up to 4 live games, works flawlessly. Of course don’t know how they did on start up.

        1. NBC has offered live streaming of the SNF for years, every time with multiple camera feeds at the same time (I think they are up to 8 or so now) and multiple audio streams, well not exactly like what F1TV has to deal out, but they change location every week and never miss a beat. Even when it first launched it had very few problems (nothing on the scale of F1TV). The problem with F1TV is that they hyped everything up promised the moon and then fell flat on their face. Assumption or not I think a lot of people thought it would be here for Australia after the announcement over the off season that it would be here this season, then they mention they are delaying the roll out to stress test things, when they do roll it out it quickly becomes apparent that whatever ‘stress testing’ they did wasn’t enough.

          They should have rolled it out has a free trial, clearly marketing it has a beta and with problems to be expected, people would have been happy to try it early and for free, no one would be upset that they got ripped off since they didn’t pay for anything and they may have gained some more subscribers from people that would check out a free stream but aren’t that interested in paying to check it out.

          Also don’t get me started on the design and lack of content, pretty sure Porsche Super Cup, F2 and GP3 were all promised along with F1 coverage, its been months since the first F2 race and week+ for others and not even a mention of it on the website. Add to that the poor spoilers (whoever thought a thumbnail of the podium was a good idea for a video right on the front page.

          It’s just so poorly executed and then they expect a premium for it, I understand the levels of technicality behind what they are trying to do, but like others have mentioned other sports have been doing similar things (maybe not to the same scale or degree) with no problems and little problems even from the beginning.

  2. I live in the United States. I was considering subscribing to F1 TV so I could watch races during times when I can’t be home at the TV. Then I realized that the system only works on a PC. The F1 website indicates that iOS and Apple TV versions are in the works but provides no release schedule. Given the technical problems that F1 are having with the live streams, I think I’ll wait until next year to subscribe.

    1. You can watch in Safari on iOS and use AirPlay to send it to your TV. I did that last night.

  3. The replay streams are fine, though they do put the podium finishers in the video description for any archived (not on the front page) race, which is irritating.

    They need to add more archive races, and they need the videos to remember the last playback position, because stopping the video and returning to it later starts it over.

    Also none of Sky’s pre-race or post race analysis is accessible either live or archived, which is understandable but unfortunate.

    And yeah, the live stream is unusable at the moment, though they did give me a partial refund equal to one month free.

  4. F1 seems to have many teething problems at the moment, like this weekend there were moments where the drivers speed was listed in kph, and it was obvious the the kph letters should have been written mph. Also a few times this year, on website and in coverage kubica has been listed as Williams driver for race. Another funny one was the pit wall classification tower in Spain. Instead of KUB for kubica they wrote 40.

  5. F1 TV presents a significant technical challenge. It involves broadcasting two dozen live streams in 1080p HD quality from a range of different international locations, while also ensuring the footage can only be accessed in certain regions.

    FOM has enlisted the services of several different suppliers to deliver F1 TV. The problems it has experienced are understood to involve making those technologies work together seamlessly to deliver live playback.

    Speaking as an IT Incident Manager, this sounds like an absolute nightmare. In an ideal world you’d want the entirety of your service mapped, end to end, with complete monitoring and logging in place. With that, you can then follow the data and see where you start to lose transactions, which will generally give you a very precise idea of where the problem occurs. If you’re dealing with multiple suppliers at different points in the chain, likely in different timezones with different monitoring systems, getting that kind of clarity is almost impossible. And I’ll say from experience that no single supplier is likely to want to carry the can for it, especially given the financial implications. I’m glad it isn’t my mess to sort out!

    1. To be honest it seems the core platform is good, the issues only become prevalent on mass rollout for live streaming.

      If only this world was as simple as giving more bandwidth. Sadly times have moved on and it is likely an interoperability error with one of the suppliers which only shows at high demand times.

      They will have the logging and metrics to find the issue, the question is whether they do a workaround or a fix.

    2. When I was looking for how to solve issues I had with an online game I played not running / not running smoothly, i got a bit of a look into what all comes into play and how different the end user experience can be even with people who are “right next to each other” because one gets routed through a different provider and interconnectivity and routing between providers, routers, servers etc can vary I can only imagine how tough streaming HD content is @spoutnik, @mazdachris.

      If it only affects some of the viewers, I think Liberty is already doing a better job than almost all of the F1 teams who did live streamed launch events in the last 2 years together!

      I guess there really aren’t that many platforms that are able to do this right now, so Liberty is building something that will be a huge asset for them if they do manage to get it working smoothly.

      1. I think their major problem affect all the users. On paper the system is scalable to handle millions of users but the trickiest part lies in the fact that millions of viewers arrive at the very same moment. As for being “right next to each other” fortunately it is not the case as regional backbones have fixed maximum capacity and it would not be possible without infrastructures upgrades (more optic fibers).

        So that it is a distributed service is a good thing. But indeed it is a very ambitious program and I can tell the challenge is already high where I work, with massive peaks (elections, attacks, world cup, ..) and it is only a Belgian media.

        1. @bascb in reply to your post

        2. I was wondering about “some viewers/customers” already yeah @spoutnik. Makes sense that every peak is an issue, and with potentially a hundred million viewers worldwide tuning in on to the same service from a range of regions and using a multitude of devices, it can not be easy to manage smoothly.

          As you mention, this is clearly something that is a building block of how our media content will be brought to us (or in your case, how you will be bringing us your content!).

    3. @mazdachris – I come to this site to escape from the words “IT incident” in my profession. Don’t go about typing that here and spoiling my day. That said, hoo boy, I wouldn’t mind being party to the ITSM update emails :-)

      I’m sure that in very short order they already had the diagnostic data to identify the problem areas, and they just didn’t have enough time & resources to fix that prior to the race, hence the decision to refund a fortnight’s worth of subscription.

    4. @mazdachris @spoutnik @bascb @m-bagattini @keithcollantine @ivan-vinitskyy – all valid problems yet when it is all said & done, the main issue is from the very outset they went with the wrong people. and with an ownership change in-between – that only adds one problem onto another and so on. at this point, it`s like unraveling a hundred miles of wool that was knitted into the world`s most complicated sweater by a group of people that only really knew who to know and how to get the job rather than how to actually do the job.

      1. Fast, what you say doesn’t really make sense. For 1. I think there IS no “wrong people” since what they are doing is pretty much a first in the industry. There might be a few parties that are better prepared to pull it off (google, anyone else?), but that would mean just exchanging one party you are reliant upon (broadcasters) for another.

        And even more amiss is the idea of “an ownership change in between”, when the whole idea of this service only started once Liberty took over FOM. Bernie had burned his fingers with the digital subscription ages ago and was reluctant to have anything to do with “the internets”

        1. @bascb – your reply is inaccurate. although to be fair – it does serve to highlight that you don’t have the same access to information and that you just like to chat online about things that you don’t know about. i’ve said more than enough – those of you who are in the know, know and those of you who don’t, don’t.

          1. So if I don’t know what you know – please enlighten the change of ownership that apparently happened in between. Since it is not the one with Liberty, which one? At Tata? or where.

            And I am sure that it is quite possible that Liberty did not choose the best partners to do this, but it is hard to know they had the chance to pick other partners that would make it work better. Maybe you know this is the case (you are right, I have none such information), but unless you give us any good reason to trust you DO know more, why would we believe you?

          2. Formula 1 is now capable of delivering broadcasts directly to internet users following tests carried out in 2016.
            Work carried out by Tata Communications in conjunction with Formula One Management, which included a test run at last year’s Singapore Grand Prix, has proved that the technology is now in place for ‘Over the top’ broadcasts, more commonly known as ‘OTT’.

            https://www.autosport.com/f1/news/128099/f1-now-capable-of-internet-broadcasts

  6. “According to FOM the problems it encountered last weekend “only affect customers during live playback.” ”
    Well that’s good, I guess most subscribers only really want to watch it replayed.

    1. For the most part, yes. I like sleeping in on Sunday.

  7. I wanted to use this service, but it says not available in my affluent country (Australia) seriously? Heaps of pubs show sky sports live on big screens, but why this not available? It’s no third world nation.

    1. sorry kpcart, nothing to do with how rich a country is or where it sits on the development ranking (if it were like that then Bernie would have introduced it).

    2. My country isn’t offered either (Bermuda), although nearly all the other Caribbean islands are. We get a mash up of feeds from Jamaica and Latin America (tv shows in English but commercials in Spanish). Would be nice of them to add us to the list! So far they have only aired one live race on ESPN which was the last one.

  8. How about simply not charging for it until they know IT WORKS??

    1. geargrinder
      15th May 2018, 14:58

      Because that would cause even MORE people to try it and further overload the system. The problem appears to be bandwidth, not the stream itself.

  9. It wouldn’t take my payment for a few days. Finally got it to process right before quali. I missed Q1 & Q2 due to buffer issues. For some reason Q3 worked pretty well. Was able to access the on demand within the hour of completion. Didn’t try the live race feed as it was Mother’s day. On demand worked great that night. If they keep improving and adding archived races; watched the 2009 German GP last night, I will be happy with the $90 (maybe less??) annual cost.

  10. Most technologies that work similar to this at launch have all most identical problems, the problem is usually scaling and provisioning of the servers to meet the demand globally. Testing and live usage very very rarely get it right first time anymore.

    It sucks, but is almost expected nowadays for a new product of this type; now they know what needs to happen i’d expect it to be much better next time around. They took an awful lot on for a first delivery, I would have done the fewer feeds approach and had others as coming soon rather than trying to cover everything at once.

    Those of us in the IT world or the even the Gaming world or any service that uses high global server usage at the start normally sucks.

    1. @skettlewood – Liberty appear to have made a decision to do a big bang launch, rather than a phased rollout, which would have better helped identify issues without causing as much angst.

      A phased approach wouldn’t work for online gaming (since day 1 is paramount), but that wouldn’t be as problematic for a racing season spread across a year.

      Even from a marketing perspective, the could have spun the marketing campaign for a phased rollout in an attractive manner – base it on an invite-based system, hyping it up as “you’re invited to exclusive access to onboard views”, thereby controlling the number of users being brought onboard. It worked pretty well for Gmail, and in its early days it did help build buzz around people getting invites to sign up for an account.

      I read elsewhere that F1 TV did have a beta period, but I’m not sure if that was focused on testing only functionality, or stress testing as well. It does feel like it was a closed beta, since I’d not heard of it before.

  11. On quite another note than most comments so far – shouldn’t it be considered a huge step that they seem to be ready to give partial refunds to customers who are affected, seemingly without too much deliberating. To me that is a huge positive.

  12. Can you imagine Bernie offering a refund?

    1. Can you imagine Bernie even offering streaming?

  13. 1080p 20 streams, that is a lot of bandwith. Even YouTube does not work always.

    1. Youtube with 1080p 200000 streams?

  14. Hemingway (@)
    15th May 2018, 15:59

    I imagine Bernie thinks this is hilarious 😂

    1. @theessence Bernie still doesn’t know what the internet is..

  15. Not spending a dime on Halo cams.

  16. Talking about failed commercialism. Can this site be fixed please? I’m fine with some ads on a site, but the way this site is flopping around like a fish on dry land for ages, before it finally displays the page properly, is simply embarassing.

    1. Mickey's Miniature Grandpa
      15th May 2018, 18:15

      I don’t hear anyone else complaining. Maybe your browser and/or connection setup need sorting out. It all works fine for me, fast and smooth.

    2. if you have particularly annoying ads, I am sure Keith will be interested in getting info on that (sent to the contact email) @patrickl. I have sent him screenshots / links etc in the past and he is quite active in getting rid of ads that are too intrusive and do not follow the site’s policy in the past.

      1. @bascb It’s not the ads themselves that bother me, but the rebuilding of the page while displaying the ads (I guess). Yesterday it was transforming itself in 5 or 6 steps or so till it was done. Text sliding down, changing in in width. Then again another width. Then again a bit down. etc etc. A while later it would slide down again.

        Trying to read something that keeps moving around was quite annoying.

        Not something that translates well into a screenshot :)

        It stopped doing that now though, so I guess it got fixed.

        1. Well, glad to hear it seems to work better for you now @patrickl! Let’s hope that state continues ;-)

  17. Is it still on a limited roll-out, because I signed up to get it and heard nothing back. Or is there some other way to get F1TV?

  18. Mickey's Miniature Grandpa
    15th May 2018, 18:24

    Teething troubles of this nature were to be expected with F1TV. It’s a much bigger undertaking than a lot of people might think and such a full-featured, early launch was optimistic if not naive. If they can get it working consistently by the end of the year, job’s a good ‘un.

    1. There are plenty of other sports streaming services that work. F1 decided to do it on their own instead of using a proven platform and this is what they get.

      They also chose to roll it out on a race weekend when it was “ready” (it isn’t) instead of testing it early on during a practice session where they could have identified these issues.

  19. Tata have the fibre from the race, but I don’t think they’re the company to be distributing it to the internet – that takes much more infrastructure.

    Hopefully they learn from other digital providers quickly – NBA, NFL… even BT Sport are doing a decent (and better!) job digitally!

  20. The new owners of the Formula 1 have shown that they are a sloppy, little professional with this failed attempt of their F1 TV.
    This suggests that the future of Formula 1 will be getting worse in every way.

  21. Looks like I’m sticking to iptv for the time being then.

  22. motogp has had its Videopass for 5 years. It does have 6 feeds (which 4 can be viewed simultaneously) all in 1080p and works. Not the 20 of F1 perhaps but then 5 years in the tech game is like 2 lifetimes so the likelihood that there is insufficient technology to offer 20 streams today is pure fantasy. Bandwidth is also not an issue unless the developers have not provided sufficient and there is no excuse for that as the number of users is well known, being a subscription service. It’s down to poor design, poor testing and poor implementation.

  23. I have subscribed to the basic subscription model which gives the archive. I am very pleased in almost every respect. The major issue is that each archived race lists the podium finishers right next to the play button. It is very difficult to scroll through an entire season and cover the bit which lists the podium finishers with my hand. If that were removed, or was made optional, then I would have nothing but praise for service, but as things stand it is a major disincentive to watching pretty much anything but the races from the last few years.

  24. Kenny Schachat (@partofthepuzzle)
    17th May 2018, 1:06

    I’ll state one thing categorically: if F1 TV Pro is using any part of ESPN streaming technology than they are in for a world of pain. Based on years of experience: ESPN live streaming video is constantly plagued stuttering, glitches, buffering resolution downgrades and other gremlins. It doesn’t matter what device or OS, you will have problems more often than not. It’s gotten so bad that I don’t even bother watching it. I’ll have a better viewing experience with a pirate stream.

    I signed up for F1 TV Pro and watched the Spanish GP on replay. It was OK for most of the race. The commentary audio cut out for a couple of minutes during the race, too. Seems like I made the right decision to skip the live stream. I would have been really annoyed if I had some the problems that other had during the live race

Comments are closed.