← Back

Overload methods with parameter lists

Have you ever wondered if you can enrich a method that you need to implement in a class to get more information from the call site?

For example, let’s say you have method debug in a logger interface AbstractLogger. Can we implement the logger interface and at the same time overload debug with another version that takes more parameters every time the users of our API call debug?

In fact, can we do this without breaking binary compatibility in the interface that defines debug and ensuring that the users of our API can only call the enriched method?

I asked myself this question three days ago and I came up with a solution that I think it’s worth a short explanation in this blog post. My use case was triggered by a feature I wanted to add to bloop (a fast compilation server for Scala).

What is the Problem?

It’s always amazed me how verbose the debug log level can be under tools such as sbt. This verbosity typically stands in the way of finding the cause for a resolution or compilation misbehavior. Running debug on the sbt shell would dump more than 20.000 logs in my screen, enough to overflow my terminal buffer and lose potentially important debug logs on the way.

I’ve found myself often in this scenario. It feels like you’re trying to find a needle in a haystack. It can be better if you’re lucky enough to know the shape of the debug messages you’re after (you can grep), but this is rarely the case.

I wanted bloop users to have a better time debugging a compilation or testing issues by narrowing down the scope of the debug logs with filters. bloop test my-app --debug test would only dump debug logs related to the test task, instead of all the other debug messages in unrelated tasks.

The logging infrastructure in bloop implements several third-party Logger interfaces and aggregates them in an abstract class BloopLogger (for simplicity we’ll extend only one: AbstractLogger).

// A third-party logger interface (in our classpath)
abstract class AbstractLogger {
  def debug(msg: String): Unit
  def info(msg: String): Unit
  def error(msg: String): Unit
}

// The logger interface that we use in all the bloop APIs
abstract class BloopLogger extends AbstractLogger

// One simple implementation of a bloop logger
class SimpleLogger extends BloopLogger {
  override def debug(msg: String): Unit = println(s"Debug: $msg")
  override def info(msg: String): Unit = println(s"Info: $msg")
  override def error(msg: String): Unit = println(s"Error: $msg")
}

We’d like to add an enriched version of debug that looks like debug(msg: String)(implicit ctx: DebugContext), where DebugContext identifies the context where debug is called. (We decide to make the parameter implicit, but there’s no reason why you shouldn’t be able to make it explicit.)

Third-party logging APIs are frozen and cannot be modified, so we cannot change the original debug method signature in AbstractLogger.

Besides, we don’t want to add an special method debugWithFilter that we would need to teach to all Bloop project contributors. We would spend a lot of time telling contributors that they must not use the normal debug, but debugWithFilter because bla.

What we really want is to shadow the original debug method with the “enriched” debug method so that only the latter can be used by default.

So, wrapping up, we don’t only want to overload a method, but also shadow it, and we want to do that without changing the public API of the interfaces we implement. It looks hard but let’s persevere.

How do we go about implementing this feature?

A First Approach

We know beforehand the compiler will tell us there’s some kind of ambiguity between the two debug methods, but bear with me and let’s write the simplest possible solution: let’s add the new debug method in SimpleLogger and then use it in a small main method.

// The debug context that we want to pass implicitly
sealed trait DebugContext
object DebugContext {
  case object Ctx1 extends DebugContext
  case object Ctx2 extends DebugContext
}

// The logger interface that we use in all the bloop APIs
abstract class BloopLogger extends AbstractLogger {
  def debug(msg: String)(implicit ctx: DebugContext): Unit
}

// One simple implementation of a bloop logger
final class SimpleLogger extends BloopLogger {
  override def debug(msg: String): Unit = println(s"Debug: $msg")
  override def info(msg: String): Unit = println(s"Info: $msg")
  override def error(msg: String): Unit = println(s"Error: $msg")

  override def debug(msg: String)(implicit ctx: DebugContext): Unit =
    println(s"Debug: $msg")
}

// The application that is the use site of our logger API
object MyApp {
  def main(args: Array[String]): Unit = {
    val logger = new SimpleLogger
    logger.debug("This is a debug message")
  }
}

When we compile the above code, the compiler emits the following error:

ambiguous reference to overloaded definition,
  both method debug in class SimpleLogger of type (msg: String)(implicit ctx: DebugContext)Unit
  and  method debug in class SimpleLogger of type (msg: String)Unit
  match argument types (String)

(Scastie link to runnable code here.)

Can we work around this ambiguous reference? Let’s consider all our possibilities.

If we try to change the call-site to select the most specific debug method (logger.debug("This is a debug message")(DebugContext.Ctx1)), the error persists.

If we try to move the new debug definition to the implementation class, then it won’t be usable by APIs using BloopLogger, which the rest of our codebase does because we have several implementations.

It looks like everything is lost. But this is the moment when knowing or intuiting that the ambiguity checker inside the compiler relies on the linearization order saves your day.

A Solution that Relies on Class Linearization

First off, what’s the linearization order? There are a few good resources in Internet, such as this one, that explain it well. But let me oversimplify and say that you can think of the linearization order as the order with which Scala will initialize an instance of a given class and all its super classes (or traits).

When Scala looks up the definition of a member, it relies on the linearization order to pick the first unambiguous candidate. A quick example:

trait A
trait B extends A { def foo: String }
trait C
class D extends C with B

// The linearization order to find `foo` is `D -> C -> B`
(new D).foo

The same procedure happens when Scala checks for ambiguous references and emits errors such as the ones we got before. As this example illustrates, the compiler will not exhaustively look for definitions of foo in all transitive super classes, it stops at the first search hit.

This insight means that we can modify our previous example such that our enriched debug method is always found first. This way, we dodge the ambiguous reference to overloaded debugs.

// We make `DebugLogger` private at the logging package level to avoid undesired users
private[logging] abstract class DebugLogger extends AbstractLogger {
  protected def printDebug(msg: String): Unit
  override def debug(msg: String): Unit = printDebug(msg)
}

// The logger interface that we use in all the bloop APIs
abstract class BloopLogger extends DebugLogger {
  def debug(msg: String)(implicit ctx: DebugContext): Unit
}

// One simple implementation of a bloop logger
final class SimpleLogger extends BloopLogger {
  override def info(msg: String): Unit = println(s"Info: $msg")
  override def error(msg: String): Unit = println(s"Error: $msg")
  override protected def printDebug(msg: String): Unit =
    println(s"Debug: $msg")
  override def debug(msg: String)(implicit ctx: DebugContext): Unit =
    printDebug(s"$msg ($ctx)")
}

The trick to make the previous code work is defining the implementation of the simple debug method in DebugLogger and making BloopLogger extend DebugLogger. We have introduced a printDebug to avoid inter-dependencies between the two debug methods, as they will cause other reference errors.

Once we have defined the method we want to shadow in a super class of the class we want to support in our API (in this case BloopLogger), the logger implementations only need to define the enriched debug method.

Users of this API will not be able to call the simple debug unless they do an upcast to the third-party logger AbstractLogger. This is intended – the goal is to have a good default, not to make it completely impossible to call the simple debug, so make sure that it still has a sensible implementation.

In Bloop’s case, the third-party logger API is never exposed and I recommend doing so in your application or library if you can.

With the above code, compiling our simple MyApp fails compilation with a could not find implicit value for parameter ctx: DebugContext, which confirms us that Scala is successfully selecting the right method.

We can fix it by passing the context either implicitly or explicitly.

object MyApp {
  def main(args: Array[String]): Unit = {
    println(
      "Running demo application for https://jorge.vican.me/post/overload-methods-with-more-parameter-lists/")
    implicit val ctx: DebugContext = DebugContext.Ctx1
    val logger = new SimpleLogger
    logger.debug("This is a debug message")
  }
}

Complete Scastie Example

Conclusion

Overloading a method inherited from a third-party class is possible in Scala. It requires a little bit of gymnastics, but once we’re familiar with the technique we can apply it in many other scenarios.

The same technique works with new explicit parameters (instead of implicit parameters). They key point is that we can overload methods by adding extra parameter lists to their definition, playing with the linearization order and defining the methods in the right place.