All posts by derny

Email Notification and Webhook Services Are Now Live!

We have great news for those of you who work with our CSV’s and/or MySQL databases.

Pro and ProPlus subscribers can now benefit from the following new services, effective November 1st, 2018.

Email Notifications.

Manually logging in to retrieve the latest zip packages is certainly not the best use of your time. Starting Nov 1st, you can sign up for automated emails that will arrive within seconds after zips have been updated. Best of all, these emails will contain a link to download.

Webhooks

Developers can now automate the process of updating MySQL databases using Webhooks.

Each webhook is a HTTP POST request to the specified URL. The request will include two items:

  • A Header which will contain a signature calculated from the secret key for the customer
  • A JSON payload

There is no authentication provided by the webhook system, your endpoint must be open, but the signature can be used to verify the request. While the url isn’t required to use https, it is recommended over http to be secure.

For more details, consult the Webhook documentation:

AA_Webhook_Documentation

Interested in either of these services? Send us an email and we will get you setup!

 

NCAA College Football Stats Now Available!

Take your draft analysis to the next level with our expanded dataset that now links over 7,000 current and former NFL draft-picks and players with their statistics from college.

A College.csv is available showing year-by-year totals, linked to our other tables with the ArmchairAnalysis.com player ID. We currently track college stats for 219 different institutions.

Our API has also been updated. Click the link below to see a sample response:

https://armchairanalysis.com/api/1.0/test/player/AA-0900/college

In terms of potential for research, we can safely say this is a huge update!

 

2018 Game Charters Required!

We are always looking for talented and committed people that know football and love to watch football each and every week of the season.

What do you get in return? Well, I’m glad you asked!

As a valued game-charter, you will receive free access to all the data we provide, both in CSV format and via our comprehensive API. You will also have instant access to the charted data as it is entered on a weekly basis during the season, weeks before it is repackaged for the general public. We also give you money towards your NFL GamePass subscription – typically $100US.

You will be responsible for the same team(s) from the start, allowing you to gain an intimate knowledge of every aspect of their offense and defense. This is going to be important as the program evolves! We do not chart pre-season games; however, the 11 playoff games are charted.

Please reach out to us via the contact form for more details on what we chart and how the program works.

 

Behind the Numbers: Building a Virtual NFL Statistician

A project 13+ years in the making

Without a doubt, one of the more common questions we field here is: “Where do you get your data?”

It’s a valid question on a number of fronts. The first comes from a legal perspective. Are we entitled to distribute this data? Does the NFL not hold the rights to data from it’s games? Beyond the legal rights issue, some people are simply wondering how accurate the data is in comparison with official league records.

The best place to start is with the following:

pbp_csv

This, my friends, is what the sporting-world refers to as a “play-by-play”. It doesn’t just appear out of thin air. League statisticians from every NFL stadium are tasked with watching plays as they unfold and marking down the details of each and every one. This information then makes it’s way from individual stadiums and within seconds of play completion, can be found on dozens of sites worldwide.

This small snippet of information is all we need to perform our magic. Nothing more, nothing less. A game might feature 160 such snippets. A week might total something in the neighbourhood of 2,500. Numbers aside, it’s what we do with these freely-available snippets of text that really set’s our site apart from the other unofficial sources of NFL data.

Relying on nothing more than play-by-play text means we don’t need to illegally scrape sites for data or rely on any other service to provide us with what we need. While sports leagues have tried to copyright basic written accounts of sporting events, they haven’t had the best of luck. Reporting on the number of times Brett Favre passed to Antonio Freeman in his career based on game accounts is no more of an infringement on copyright than reporting on the number of times a character spoke Harry Potter’s name in a J.K. Rowling book (16,612 if you are interested).

The real question is, how do we get from this:

pbp_csv

To something like this:

{
  "data": {
    "gid": 3990,
    "pid": 651831,
    "off": "PIT",
    "def": "NE",
    "type": "RUSH",
    "dseq": 1,
    "len": 34,
    "qtr": 1,
    "min": 7,
    "sec": 41,
    "ptso": 0,
    "ptsd": 0,
    "timo": 3,
    "timd": 3,
    "dwn": 1,
    "ytg": 10,
    "yfog": 7,
    "zone": 1,
    "fd": 0,
    "sg": 0,
    "nh": 0,
    "pts": 0,
    "tck": [
      {
        "uid": 430825,
        "tck": "GG-0475",
        "value": "0.5"
      },
      {
        "uid": 430826,
        "tck": "AB-2700",
        "value": "0.5"
      }
    ],
    "sk": 0,
    "pen": 0,
    "ints": 0,
    "fum": 0,
    "saf": 0,
    "blk": 0,
    "olid": {
      "lt": "KB-0750",
      "lg": "AV-0350",
      "c": "RF-0900",
      "rg": "CH-4550",
      "rt": "CW-0400"
    },
    "rush": {
      "bc": "DW-3600",
      "dir": "LG",
      "yds": 6,
      "succ": 1,
      "kne": 0
    }
  }
}

And even this:

2017 NFL Database Field Listing

Going from point A (play-by-play text) to points B, C, D and beyond required us to do 2 things:

  1. Build a ‘virtual’ NFL statistician able to digest those simple play-by-play accounts to create player and team statistics and everything else in-between.
  2. Hire our own real live ‘Statisticians’ to analyze game tape to fill in the holes and also expand on the basic play-by-play data.

The first task has taken over a decade (of off and on work) and probably in the neighborhood of 15,000 lines of code to achieve. This is not an exaggeration. Here is what’s required to properly parse the effect of Fumbles:

Sub FUMBLES()

If S1.Cells(ROW1, 6) = "PASS" Or S1.Cells(ROW1, 6) = "RUSH" Then
   X8 = InStr(X7, BOOK.Cells(X2, 2), "FUMBLES") + 8: X11 = 2
   If S1.Cells(ROW1, 26) <> "" Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 26)
   If S1.Cells(ROW1, 35) <> "" And InStr(X7, BOOK.Cells(X2, 2), "FUMBLES") > InStr(X7, BOOK.Cells(X2, 2), "pass") Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 35)
   If InStr(X7, BOOK.Cells(X2, 2), "FUMBLES (Aborted)") > 0 Then
      For X3 = 1 To 6
      If QBNAME(X3, X6 + 1) = "" Then Exit For
      If InStr(Mid(BOOK.Cells(X2, 2), X7, 40), QBNAME(X3, X6 + 1)) > 0 Then S1.Cells(ROW1, 61) = "QB " & QBNAME(X3, X6 + 1): Exit For
      Next X3
      If S1.Cells(ROW1, 61) = "" Then
         For X3 = 1 To 75
         If ALLNAME(X3, X6 + 1) = "" Then Exit For
         If InStr(Mid(BOOK.Cells(X2, 2), X7, 40), ALLNAME(X3, X6 + 1)) > 0 Then S1.Cells(ROW1, 61) = ALLPOS(X3, X6 + 1) & " " & ALLNAME(X3, X6 + 1): Exit For
         Next X3
      End If
   End If
   If InStr(X7, BOOK.Cells(X2, 2), "Aborted") > 0 And InStr(X7, BOOK.Cells(X2, 2), "FUMBLES (Aborted)") = 0 Then
      For X3 = 1 To 75
      If ALLNAME(X3, X6 + 1) = "" Then Exit For
      If InStr(Mid(BOOK.Cells(X2, 2), InStr(X7, BOOK.Cells(X2, 2), "Aborted") + 8, 25), ALLNAME(X3, X6 + 1)) > 0 Then S1.Cells(ROW1, 61) = ALLPOS(X3, X6 + 1) & " " & ALLNAME(X3, X6 + 1): Exit For
      Next X3
   End If
   If InStr(X7, BOOK.Cells(X2, 2), "Lateral to") > 0 And InStr(X7, BOOK.Cells(X2, 2), "FUMBLES") > InStr(X7, BOOK.Cells(X2, 2), "Lateral to") Then
      For X3 = 1 To 75
      If ALLNAME(X3, X6 + 1) = "" Then Exit For
      If InStr(Mid(BOOK.Cells(X2, 2), InStr(X7, BOOK.Cells(X2, 2), "Lateral to") + 11, 25), ALLNAME(X3, X6 + 1)) > 0 Then S1.Cells(ROW1, 61) = ALLPOS(X3, X6 + 1) & " " & ALLNAME(X3, X6 + 1): Exit For
      Next X3
   End If
   If S1.Cells(ROW1, 61) = "" Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 31)
End If

If S1.Cells(ROW1, 6) = "KOFF" Or S1.Cells(ROW1, 6) = "PUNT" Or S1.Cells(ROW1, 6) = "ONSD" Then
   If S1.Cells(ROW1, 6) = "PUNT" Then X11 = 2 Else X11 = 1
   If InStr(X7, BOOK.Cells(X2, X11), "MUFFS") > 0 Then
      X8 = InStr(X7, BOOK.Cells(X2, X11), "MUFFS") + 6
   Else
      X8 = InStr(X7, BOOK.Cells(X2, X11), "FUMBLES") + 8
   End If
   If InStr(X7, BOOK.Cells(X2, X11), "FUMBLES (Aborted)") > 0 Then
      If S1.Cells(ROW1, 72) <> "" Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 72)
      If S1.Cells(ROW1, 79) <> "" Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 79)
   End If
   If InStr(X7, BOOK.Cells(X2, X11), "Aborted") > 0 And InStr(X7, BOOK.Cells(X2, X11), "FUMBLES (Aborted)") = 0 Then
      If S1.Cells(ROW1, 6) = "PUNT" Then X10 = X6 + 1 Else X10 = 2 - X6
      For X3 = 1 To 75
      If ALLNAME(X3, X10) = "" Then Exit For
      If InStr(Mid(BOOK.Cells(X2, X11), InStr(X7, BOOK.Cells(X2, X11), "Aborted") + 8, 25), ALLNAME(X3, X10)) > 0 Then S1.Cells(ROW1, 61) = ALLPOS(X3, X10) & " " & ALLNAME(X3, X10): Exit For
      Next X3
   End If
   If InStr(X7, BOOK.Cells(X2, X11), "Lateral to") > 0 And InStr(X7, BOOK.Cells(X2, X11), "FUMBLES") > InStr(X7, BOOK.Cells(X2, X11), "Lateral to") Then
      If S1.Cells(ROW1, 6) = "PUNT" Then X10 = 2 - X6 Else X10 = X6 + 1
      For X3 = 1 To 75
      If ALLNAME(X3, X10) = "" Then Exit For
      If InStr(Mid(BOOK.Cells(X2, X11), InStr(X7, BOOK.Cells(X2, X11), "Lateral to") + 11, 25), ALLNAME(X3, X10)) > 0 Then S1.Cells(ROW1, 61) = ALLPOS(X3, X10) & " " & ALLNAME(X3, X10): Exit For
      Next X3
   End If
   If InStr(X7, BOOK.Cells(X2, X11), "punt is BLOCKED") > 0 And InStr(X7, BOOK.Cells(X2, X11), "FUMBLES") > InStr(X7, BOOK.Cells(X2, X11), "punt is BLOCKED") Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 67)
   If S1.Cells(ROW1, 61) = "" Then
      If S1.Cells(ROW1, 6) = "PUNT" Then S1.Cells(ROW1, 61) = S1.Cells(ROW1, 76) Else S1.Cells(ROW1, 61) = S1.Cells(ROW1, 83)
   End If
End If

If InStr(X8, BOOK.Cells(X2, X11), "recovers at") > 0 Then S1.Cells(ROW1, 62) = S1.Cells(ROW1, 61)
If InStr(X8, BOOK.Cells(X2, X11), "recovered by") > 0 Then
   If S1.Cells(ROW1, 6) = "PUNT" Then X10 = 2 - X6 Else X10 = X6 + 1
   For X3 = 1 To 75
   If ALLNAME(X3, X10) = "" Then Exit For
   If InStr(Mid(BOOK.Cells(X2, X11), InStr(X8, BOOK.Cells(X2, X11), "recovered by") + 13, 25), ALLNAME(X3, X10)) > 0 Then S1.Cells(ROW1, 62) = ALLPOS(X3, X10) & " " & ALLNAME(X3, X10): Exit For
   Next X3
End If

If InStr(X8, BOOK.Cells(X2, X11), "RECOVERED by") > 0 Then
   If S1.Cells(ROW1, 6) = "PUNT" Then X10 = X6 + 1 Else X10 = 2 - X6
   For X3 = 1 To 75
   If ALLNAME(X3, X10) = "" Then Exit For
   If InStr(Mid(BOOK.Cells(X2, X11), InStr(X8, BOOK.Cells(X2, X11), "RECOVERED by") + 13, 25), ALLNAME(X3, X10)) > 0 Then S1.Cells(ROW1, 62) = ALLPOS(X3, X10) & " " & ALLNAME(X3, X10): Exit For
   Next X3
   If S1.Cells(ROW1, 62) <> "" Then
      X9 = InStr(X7, BOOK.Cells(X2, X11), "RECOVERED by") + 13
      If InStr(Mid(BOOK.Cells(X2, X11), X9, 90), "for no gain") + InStr(X9, BOOK.Cells(X2, X11), "Touchback") > 0 Or InStr(Mid(BOOK.Cells(X2, X11), X9, 90), " for ") = 0 Then S1.Cells(ROW1, 63) = 0
      If S1.Cells(ROW1, 63) = "" Then
         If IsNumeric(Mid(BOOK.Cells(X2, X11), InStr(X9, BOOK.Cells(X2, X11), " for ") + 7, 1)) = True Then S1.Cells(ROW1, 63) = Mid(BOOK.Cells(X2, X11), InStr(X9, BOOK.Cells(X2, X11), " for ") + 5, 3) Else S1.Cells(ROW1, 63) = Mid(BOOK.Cells(X2, X11), InStr(X9, BOOK.Cells(X2, X11), " for ") + 5, 2)
      End If
   End If
End If

End Sub

Building a customized ‘rules engine’ that produces player and team data with 100% accuracy means accounting for every possible event on a Rush; Pass; Punt; Kickoff; Field Goal; Extra Point or 2pt Conversion — no easy task.

You might wonder though: why on Earth would we take the time to build a complex system that can construct player and team statistics from nothing more than play-by-play text?

Firstly, we feel there should be no free lunch. Stealing significant portions of player and team stats from other sites is not something we condone and it’s not a viable business model long-term. The raw play-by-plays we work with would be largely useless to businesses and fantasy football players that need highly organized data in a relational database and this is where our “Sweat of the Brow” comes in, both through the thousands of lines of code required to parse play-by-plays as well as our very own Charting Program (est. in 2016) which takes things to whole other level of detail.

Secondly, diving down to the play level gives us the flexibility to analyze the data in thousands, if not hundreds of thousands of ways. It’s a level of detail you will simply not find anywhere else.

So, next time you come across a site pushing unofficial data, be sure to ask the important question, “Where do you get your data”?

Twitter Feed For All Active Players Now Available!

We’ve spent the last month working tirelessly on a new and exciting feature: Twitter content for every single player currently on each team’s 90-man roster.

As far as we know, this is the only accurate record of Twitter handles and Tweets for every active NFL player.

This data will be updated weekly during the season and is also currently available via our API. Here is a quick example of how to retrieve Aaron Rodgers recent tweets:

http://armchairanalysis.com/api/1.0/test/tweets/AR-1300

Stay tuned for yet more new content coming in the next few weeks!

 

 

 

 

Roster Updates for 2017 Starting Soon!

Welcome back NFL Fans! We are kicking off the 2017 season starting next week with the commencement of weekly roster updates.

Data on every single player currently on 90-man rosters will be available, including:

  • Combine results
  • Hand/Arm size
  • Depth Chart position

And much, much more. Data will be accessible both through the website (CSV files) and our API.

Also, we are excited to announce the inclusion of the following data in September which will help give you the full picture on QB actions and events on each play. Some of the new stats include:

  • QB Pressure – # players blitzing on each play
  • QB Flushes
  • QB Hits
  • QB Throw-aways

As well as some new data on defensive formations.

Stay tuned for more news on a few other interesting projects we are working on this off-season!

Game Charted Data is Back!

Good news NFL fans: we’ve expanded our game charting program to include a total of 16 professional charters. As a result, look for more timely updates for this specialized information this coming season.

We are currently focusing on the following data points for 2016:

  • Number of Running Backs and Tight Ends on the field (aka Personnel Groupings)
  • Yards-after-catch
  • Dropped Passes
  • Missed or broken tackles

This data will be ‘official’ for 2016 after our initial pilot project went well in 2015.

The program will be expanding to include a total of 32 game-charters in 2017. Please drop us a line via the contact form if you are interested in getting involved!

New NFL Player Stats For 2016!

There is lot’s of new stuff coming your way this off-season, starting with the addition of Arm Length and Hand Size data in the PLAYERS table. This data is now available for over 1,000 current and former players and this number is going to grow to over 2,000 in the next couple of weeks.

The PLAYERS table has also been updated to include over 700 new players that can be found on the current 90 man rosters. Look for daily updates to the player data from now until the start of the season (Hardcore and Hardcore-Lite only).

Also, many of you have been asking for a ‘Schedule’ table in order to forecast results for upcoming games in 2016. This new table is going to be available in the next week – I will send out an email when it’s ready.