eshkol

High-Performance LISP-like language for Scientific Computing and AI written in C

tsotchke
68
6
C

🚀 Eshkol

The high-performance LISP-like language that brings scientific computing and AI to the next level

MIT License
Project Status
Scheme Compatibility

⚠️ Early Developer Preview

Important: Eshkol is currently in early development. This preview release is intended for developers interested in exploring the language and potentially contributing to its development. It is not yet ready for production use.

Current Development Status

Component Status Completion Notes
Core Language Features In Progress 65% Basic Scheme syntax and core special forms implemented
Function Composition In Progress 75% Basic composition works, advanced patterns may have issues
Closure System In Progress 65% Basic closures work, global variable initialization improved
Type System In Progress 55% Optional type annotations and basic inference working
Scientific Computing In Progress 70% Vector operations and autodiff implemented
MCP Tools In Progress 80% Analysis tools available for development

See our ROADMAP.md for development plans and KNOWN_ISSUES.md for current limitations and workarounds.

🌟 What is Eshkol?

Eshkol is a revolutionary programming language that combines the elegant, expressive syntax of Scheme with the raw performance of C. Designed specifically for scientific computing and artificial intelligence applications, Eshkol delivers the perfect balance between developer productivity and computational efficiency.

Unlike other languages that force you to choose between expressiveness and performance, Eshkol gives you both:

  • Write code that reads like mathematical notation with our LISP-like syntax
  • Run at near-C speeds thanks to our direct compilation to optimized C code
  • Manage memory deterministically with our innovative arena-based memory system
  • Accelerate numerical computations using built-in SIMD optimizations
  • Differentiate functions automatically for machine learning and optimization tasks
  • Leverage existing C libraries with zero-overhead interoperability

✨ Core Features

🔥 What’s Working Now

  • Powerful Scheme Foundation - Core special forms and operations implemented:

    • define, if, lambda, begin, quote, set!, let, and, or
    • Core list operations: cons, car, cdr, list, pair?, null?, list?, set-car!, set-cdr!
    • Write expressive, functional code just like in Scheme

  • Arena-Based Memory Management - Deterministic performance without GC pauses:

    • Predictable memory allocation and deallocation patterns
    • No garbage collection pauses or unpredictable latency spikes
    • Perfect for real-time applications and high-performance computing

  • Vector Calculus Operations - Built-in support for mathematical operations:

    • Gradient computation (∇f)
    • Divergence (∇·F)
    • Curl (∇×F)
    • Laplacian (∇²f)
    • Vector field operations

  • Automatic Differentiation - First-class support for gradient-based methods:

    • Forward-mode automatic differentiation
    • Reverse-mode automatic differentiation
    • Higher-order derivatives
    • Jacobian and Hessian matrix computation (partial)

🛠️ Recently Enhanced

  • Type System - Comprehensive static typing with Scheme compatibility:

    • Gradual typing system that combines static and dynamic typing
    • Three typing approaches: implicit typing, inline explicit typing, and separate type declarations
    • Powerful type inference to reduce annotation burden
    • Compile-time type checking for early error detection
    • Deep integration with automatic differentiation and scientific computing

  • MCP Tools Integration - Development tools for analysis and debugging:

    • Type analysis tools
    • Code generation analysis
    • Binding and lambda analysis
    • AST visualization
    • Closure memory visualization
    • Mutual recursion analysis

🛠️ Currently In Development

  • Type System Improvements - Enhancing type inference and checking:

    • Type inference for automatic differentiation functions
    • Vector return type handling
    • Resolving type conflicts in generated C code
    • Integration of type information with code generation

  • Scheme Compatibility - Working towards R5RS and R7RS-small compatibility:

    • Basic type predicates
    • Additional list processing functions (append, reverse, etc.)
    • Numeric operations (partial)
    • Boolean operations (partial)

  • Scientific Computing Enhancements - Expanding numerical capabilities:

    • SIMD code generation optimization
    • Matrix algorithms
    • Array optimizations
    • Gradient-based optimization algorithms

  • Scientific Computing Primitives - Built-in support for numerical computation:

    • Vector and matrix operations with optimized implementations
    • Vector calculus operations (gradient, divergence, curl, laplacian)
    • Automatic SIMD vectorization for parallel data processing
    • High-performance mathematical functions

  • Automatic Differentiation - First-class support for gradient-based methods:

    • Compute derivatives with machine precision
    • Support for both forward and reverse mode differentiation
    • Jacobian and Hessian matrix computation
    • Perfect for optimization, machine learning, and scientific computing

  • C Interoperability - Seamless integration with existing codebases:

    • Zero-overhead FFI for calling C functions
    • Direct access to C data structures
    • Callback support for C libraries that require function pointers

  • VSCode Integration - Modern development experience:

    • Syntax highlighting
    • Language configuration
    • Enhanced editing experience

🔮 Coming Soon

  • Comprehensive Scheme Compatibility - Full implementation of R5RS and R7RS-small:

    • Phase 1 (In Progress): Core data types and fundamental operations
    • Phase 2: List processing and control flow
    • Phase 3: Higher-order functions and data structures
    • Phase 4: I/O and system interface
    • Phase 5: Advanced features

  • Advanced Concurrency - Efficient parallel computation:

    • Task parallelism for concurrent execution
    • Data parallelism for collection operations
    • Message passing for safe communication
    • Lock-free algorithms for high-performance concurrency

  • GPU Acceleration - Leverage the power of graphics processors:

    • GPGPU computing for massively parallel workloads
    • Automatic kernel generation from high-level code
    • Seamless integration with CPU code

  • Interactive Development Environment - Rapid prototyping and exploration:

    • REPL for interactive code evaluation
    • Notebook interface for literate programming
    • Visualization tools for data exploration

📊 Performance

Eshkol is designed from the ground up for high performance:

  • Compilation to C ensures optimal machine code generation
  • Arena-based memory management eliminates GC pauses
  • SIMD optimization exploits modern CPU vector instructions
  • Specialized numerical algorithms for scientific computing workloads

Early benchmarks show performance comparable to hand-optimized C code for numerical workloads, while maintaining the expressiveness of a high-level language.

🚀 Getting Started

Building from Source

mkdir -p build
cd build
cmake ..
make

Running Your First Eshkol Program

Create a file named hello.esk:

(define (main)
  (display "Hello, Eshkol!"))

Compile and run:

./eshkol hello.esk

More Examples

Check out the examples/ directory for sample programs demonstrating Eshkol’s capabilities:

Fully Working Examples:

  • hello.esk - Basic “Hello, World!” program
  • display_test.esk - Basic “Hello, World!” program using display function
  • factorial.esk - Recursive factorial calculation
  • arithmetic.esk - Basic arithmetic operations
  • tail_recursive_factorial.esk - Tail-recursive factorial implementation
  • fibonacci.esk - Fibonacci sequence calculation

Examples in Development (May Have Limitations):

  • function_composition.esk - Higher-order functions and composition
  • mutual_recursion.esk - Demonstration of mutual recursion
  • list_operations.esk - Demonstration of list operations
  • minimal_lambda.esk - Basic lambda and closure demonstration

Type System Examples (In Development):

  • untyped.esk - Standard Scheme code without type annotations
  • implicit_typed.esk - Using type inference without explicit annotations
  • inline_typed.esk - Using inline explicit type annotations
  • explicit_param_typed.esk - Using explicit parameter type annotations
  • separate_typed.esk - Using separate type declarations
  • simple_typed.esk - Simple examples of typed functions

Scientific Computing Examples (In Development):

  • vector_calculus.esk - Vector operations and calculus
  • autodiff_example.esk - Automatic differentiation in action

📚 Documentation

Comprehensive documentation is available to help you learn and master Eshkol:

Implementation Status

Vision and Planning

Type System

Tutorials and References

Scheme Compatibility

🧩 File Extensions

Eshkol uses the following file extensions:

  • .esk - Eshkol source files
  • .eskh - Eshkol header files
  • .eskir - Intermediate representation
  • .eskc - Generated C code
  • .esklib - Compiled library
  • .eskmod - Module file
  • .eskproj - Project configuration
  • .eskpkg - Package definition

🤝 Contributing

Eshkol is an ambitious project, and we welcome contributions from the community! Whether you’re interested in language design, compiler implementation, scientific computing, or AI, there’s a place for you in the Eshkol ecosystem.

Check out our Implementation Status and Scheme Compatibility Implementation Plan to see where you can help.

📜 License

MIT

📝 Citation

If you use Eshkol in your research, please cite it as:

@software{tsotchke2025eshkol,
  author       = {tsotchke},
  title        = {Eshkol: A High-Performance LISP-like language for Scientific Computing and AI},
  year         = {2025},
  url          = {https://github.com/tsotchke/eshkol}
}