Edit large files on Linux

cool_penguin_smallOne of our readers requested a list of editors on Linux capable of editing huge files in the order of GBs. In one of our earlier articles we explored some commands to create huge files on Linux. We also visited glogg, a log viewer with similar capabilities but it cannot edit files. Here’s a list of some robust editors.

1. lfhex

A Qt based GUI editor. Can view and edit files in hex, octal, binary, or ascii text mode. Can work with files much larger than system RAM or even address space.

Features

  • Low memory usage
  • Instant load times
  • Instant save times
  • Infinite undo/redo
  • Dynamic hex/octal/binary/ascii editing mode
  • Search
  • “Goto” field for jumping to a specified offset (offset can be specified by a mathematical expression: 0xff*3
  • 64 bit offset support
  • Dynamic resize support
  • Conversion dialog
    > Linked to selection
    > Shows conversion to int, float, double, ascii, hex
    > Modifying int/float/double/ascii/hex updates all the other fields
    > Option to show/edit byteswapped values
  • Binary comparison dialog
    > Differences can be walked by “block”
    > A block can be from 1-16 bytes long
    > Starting offset can be different in each file
  • Minimal dependencies (just Qt)

Limitations

  • Does not support insertion/deletion (cannot change file size)
  • Search/compare can be slow (compared to cmp or any other non-paged IO app)
  • Cannot search files with unsaved modifications

To install on Ubuntu:

$ sudo apt-get install lfhex

2. Joe

Joe s a very powerful full-featured terminal editor. Written in C and the only dependency is libc.

Features

  • Can view and edit files in text of hex mode
  • Supports UTF8 characters
  • Multi-file search and replace- file list is either given on cmdline or by a UNIX command (grep/find) run from within JOE
  • Mouse support, including wheel (works best when using xterm). The mouse can resize windows, scroll windows, select and paste text, and select menu entries.
  • Context display on status line: allows you to see name of function cursor is in
  • Syntax highlighting
  • Swap file allows editing files larger than memory
  • Bash-like TAB completion and history for all prompts
  • Jump to matching delimiter
  • and many more…

Limitations

  • NO vertical windows
  • No folding
  • No background spell checking, like Microsoft WORD
  • Cannot highlight all matching words

To install on Ubuntu:

$ sudo apt-get install joe

3. HEd

HEd is a powerful hex editor with a hexdump -C like interface. It can load and edit infinitely large files.

Features

  • Very fast on very large files (keeps only necessary portion of the file in memory)
  • Fast inserting anywhere in the file
  • Fast saving of intermediate changes
  • vim-like controls (and exmode)
  • Powerful expressions concept for flexible searching and transformation operations on the file or a selected region

HEd is not available by default on Ubuntu. Download HEd v0.5 compiled on Ubuntu 14.04 amd64 here.
md5sum: 5eb449e5d613d5925c6ee50ea11ab317

4. LargeFile

This is a plugin available for vim that turns off certain vim features to handle large files. The g:LargeFile (by default 100) option describes the minimum size of a file to be considered as a LargeFile, in megabytes. This option can be set in ~/.vimrc as:

set LargeFile=50

Limitation

Note that LargeFile may not be able to handle a 1GB file as it doesn’t change the way vim opens a file.

Installation

Download the latest version from the homepage. Then:

$ vi LargeFile.vba.gz
:so %

6 thoughts on “Edit large files on Linux”

  1. mcedit from the Midnight Commander is the easiest to use terminal editor and can edit huge files and binary, etc.

Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s