filetype

Fast, dependency-free Go package to infer binary file types based on the magic numbers header signature

Programming language: Go

License: MIT License

Tags: Utilities    

Latest version: v1.1.0

README

filetype

Small and dependency free Go package to infer file and MIME type checking the magic numbers signature.

For SVG file type checking, see go-is-svg package. Python port: filetype.py.

Features

• Supports a wide range of file types
• Provides file extension and proper MIME type
• File discovery by extension or MIME type
• File discovery by class (image, video, audio...)
• Provides a bunch of helpers and file matching shortcuts
• Pluggable: add custom new types and matchers
• Simple and semantic API
• Blazing fast, even processing large files
• Only first 262 bytes representing the max file header is required, so you can just pass a slice
• Dependency free (just Go code, no C compilation needed)
• Cross-platform file recognition

Installation

go get github.com/h2non/filetype

API

See Godoc reference.

Subpackages

• github.com/h2non/filetype/types
• github.com/h2non/filetype/matchers

Examples

Simple file type checking

package main

import (
  "fmt"
  "io/ioutil"

  "github.com/h2non/filetype"
)

func main() {
  buf, _ := ioutil.ReadFile("sample.jpg")

  kind, _ := filetype.Match(buf)
  if kind == filetype.Unknown {
    fmt.Println("Unknown file type")
    return
  }

  fmt.Printf("File type: %s. MIME: %s\n", kind.Extension, kind.MIME.Value)
}

Check type class

package main

import (
  "fmt"
  "io/ioutil"

  "github.com/h2non/filetype"
)

func main() {
  buf, _ := ioutil.ReadFile("sample.jpg")

  if filetype.IsImage(buf) {
    fmt.Println("File is an image")
  } else {
    fmt.Println("Not an image")
  }
}

Supported type

package main

import (
  "fmt"

  "github.com/h2non/filetype"
)

func main() {
  // Check if file is supported by extension
  if filetype.IsSupported("jpg") {
    fmt.Println("Extension supported")
  } else {
    fmt.Println("Extension not supported")
  }

  // Check if file is supported by extension
  if filetype.IsMIMESupported("image/jpeg") {
    fmt.Println("MIME type supported")
  } else {
    fmt.Println("MIME type not supported")
  }
}

File header

package main

import (
  "fmt"
  "io/ioutil"

  "github.com/h2non/filetype"
)

func main() {
  // Open a file descriptor
  file, _ := os.Open("movie.mp4")

  // We only have to pass the file header = first 261 bytes
  head := make([]byte, 261)
  file.Read(head)

  if filetype.IsImage(head) {
    fmt.Println("File is an image")
  } else {
    fmt.Println("Not an image")
  }
}

Add additional file type matchers

package main

import (
  "fmt"

  "github.com/h2non/filetype"
)

var fooType = filetype.NewType("foo", "foo/foo")

func fooMatcher(buf []byte) bool {
  return len(buf) > 1 && buf[0] == 0x01 && buf[1] == 0x02
}

func main() {
  // Register the new matcher and its type
  filetype.AddMatcher(fooType, fooMatcher)

  // Check if the new type is supported by extension
  if filetype.IsSupported("foo") {
    fmt.Println("New supported type: foo")
  }

  // Check if the new type is supported by MIME
  if filetype.IsMIMESupported("foo/foo") {
    fmt.Println("New supported MIME type: foo/foo")
  }

  // Try to match the file
  fooFile := []byte{0x01, 0x02}
  kind, _ := filetype.Match(fooFile)
  if kind == filetype.Unknown {
    fmt.Println("Unknown file type")
  } else {
    fmt.Printf("File type matched: %s\n", kind.Extension)
  }
}

Supported types

Image

• jpg - image/jpeg
• png - image/png
• gif - image/gif
• webp - image/webp
• cr2 - image/x-canon-cr2
• tif - image/tiff
• bmp - image/bmp
• heif - image/heif
• jxr - image/vnd.ms-photo
• psd - image/vnd.adobe.photoshop
• ico - image/vnd.microsoft.icon
• dwg - image/vnd.dwg

Video

• mp4 - video/mp4
• m4v - video/x-m4v
• mkv - video/x-matroska
• webm - video/webm
• mov - video/quicktime
• avi - video/x-msvideo
• wmv - video/x-ms-wmv
• mpg - video/mpeg
• flv - video/x-flv
• 3gp - video/3gpp

Audio

• mid - audio/midi
• mp3 - audio/mpeg
• m4a - audio/m4a
• ogg - audio/ogg
• flac - audio/x-flac
• wav - audio/x-wav
• amr - audio/amr
• aac - audio/aac
• aiff - audio/x-aiff

Archive

• epub - application/epub+zip
• zip - application/zip
• tar - application/x-tar
• rar - application/vnd.rar
• gz - application/gzip
• bz2 - application/x-bzip2
• 7z - application/x-7z-compressed
• xz - application/x-xz
• zstd - application/zstd
• pdf - application/pdf
• exe - application/vnd.microsoft.portable-executable
• swf - application/x-shockwave-flash
• rtf - application/rtf
• iso - application/x-iso9660-image
• eot - application/octet-stream
• ps - application/postscript
• sqlite - application/vnd.sqlite3
• nes - application/x-nintendo-nes-rom
• crx - application/x-google-chrome-extension
• cab - application/vnd.ms-cab-compressed
• deb - application/vnd.debian.binary-package
• ar - application/x-unix-archive
• Z - application/x-compress
• lz - application/x-lzip
• rpm - application/x-rpm
• elf - application/x-executable
• dcm - application/dicom

Documents

• doc - application/msword
• docx - application/vnd.openxmlformats-officedocument.wordprocessingml.document
• xls - application/vnd.ms-excel
• xlsx - application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
• ppt - application/vnd.ms-powerpoint
• pptx - application/vnd.openxmlformats-officedocument.presentationml.presentation

Font

• woff - application/font-woff
• woff2 - application/font-woff
• ttf - application/font-sfnt
• otf - application/font-sfnt

Application

• wasm - application/wasm
• dex - application/vnd.android.dex
• dey - application/vnd.android.dey

Benchmarks

Measured using real files.

Environment: OSX x64 i7 2.7 Ghz

BenchmarkMatchTar-8    1000000        1083 ns/op
BenchmarkMatchZip-8    1000000        1162 ns/op
BenchmarkMatchJpeg-8   1000000        1280 ns/op
BenchmarkMatchGif-8    1000000        1315 ns/op
BenchmarkMatchPng-8    1000000        1121 ns/op

License

MIT - Tomas Aparicio