Higher-order functions in Scala

This page provides information and a dataset of technical report, Understanding the Use of Higher-Order Functions: An Exploratory Study on Scala Programs.

Terminology

HOFs -- higher-order functions

1. Base Information

We selected top 35 repositories with the most stars on GitHub and extracted the definitions and calls of higher-order functions. Table 1 shows the basic information of 35 repositories

Table 1. Basic information of 35 repositories
Repository	#Star	Definitions of all functions	Calls of all functions	Definitions of HOFs	Calls of HOFs	Executable LOC	Scala files
scala/scala	12.1k	24779	96002	1216	5162	143.6k	6579
apache/predictionio	12.1k	548	1081	24	52	20.3k	259
playframework/playframework	11.3k	3095	5474	196	266	41.5k	792
akka/akka	10.3k	13654	31207	519	1292	114.8k	1013
yahoo/kafka-manager	8.0k	753	1216	9	25	10.5k	149
gitbucket/gitbucket	7.8k	1056	3218	61	543	19.1k	186
twitter/finagle	7.3k	6178	5904	157	204	63.0k	708
ornicar/lila	5.8k	5733	11240	165	505	70.3k	1517
rtyley/bfg-repo-cleaner	5.7k	119	144	10	5	1.5k	60
gatling/gatling	4.4k	2718	4186	118	345	25.6k	413
scalaz/scalaz	4.2k	8184	14594	1853	4191	35.4k	550
sbt/sbt	4.0k	4076	13638	436	1989	34.9k	360
lampepfl/dotty	3.6k	10372	41312	256	770	387.1k	17168
twitter/scalding	3.2k	4174	6285	292	846	29.6k	328
milessabin/shapeless	2.9k	1936	2416	50	46	30.5k	197
scalatra/scalatra	2.4k	1480	1823	57	33	8.4k	252
spark-jobserver/spark-jobserver	2.4k	620	698	13	30	7.6k	150
twitter/util	2.3k	2777	3278	176	500	27.4k	436
slick/slick	2.3k	2881	161	4075	391	20.8k	262
lagom/lagom	2.3k	1813	1397	70	35	21.8k	33
lihaoyi/Ammonite	2.1k	792	1211	72	133	8.6k	129
twitter/finatra	1.9k	697	590	38	14	20.2k	452
twitter/algebird	1.9k	2228	4404	230	176	22.7k	142
circe/circe	1.8k	944	778	113	68	6.9k	109
Azure/mmlspark	1.7k	1904	1960	60	87	18.0k	196
http4s/http4s	1.7k	2474	3766	160	273	29.1k	315
spotify/scio	1.7k	2122	3616	156	435	30.5k	451
sangria-graphql/sangria	1.6k	1686	3582	161	133	15.0k	104
typelevel/scalacheck	1.6k	470	1051	100	239	3.4k	27
zio/zio	1.6k	2604	5557	646	1524	13.9k	269
getquill/quill	1.5k	1414	3405	123	213	15.6k	142
tpolecat/doobie	1.5k	5672	3790	426	1485	14.7k	147
functional-streams-for-scala/fs2	1.5k	1442	2925	150	545	9.2k	81
foundweekends/giter8	1.5k	123	126	5	4	1.4k	28
ThoughtWorksInc/Binding.scala	1.4k	230	317	6	20	3.4k	8

2. Experimental Dataset

In the paper, we have two datasets, [Download datasets]. Table 2 shows the information of each dataset.

Table 2. Dataset description
Name	Content	Format
definitions	definitions of all function	csv
calls	calls of all function	csv

The following picture shows one file of definitions of all functions.

Field explanation

id: the id of functions definitions
type: type of functions; 0 represents first-order functions, 1 represents higher-order functions
file_path: relative path to the scala files
code: a sample explanation of functions
line_num: the location of the functions definitions in the source code
h_type: type of higher-order functions definitions;
           1 represents a function definition takes at least one function as a parameter without returning functions;
           2 represents a function definition returns at least one function without inputing a function;
           3 represents a function definition takes at least one function as a parameter and returns functions.
name: the full name of functions
source code: the first line of higher-order functions source code
author: the author of higher-order functions definitions
call_num: the number of times higher-order functions are called
full_line: the full line of higher-order functions source code
lineNum: the executable LoC of higher-order functions definitions
cyc: the Cyclomatic complexity of higher-order functions definitions
style: the warnings in the code style of higher-order functions definitions
num_author: the number of commits to source code of the higher-order function definitions
num_commit: the number of authors of the higher-order function definitions

The following picture shows one file of calls of all functions

Field explanation

id: the id of functions calls
dec_id: the id of the function being called
line_num: the location of the functions calls in the source code
type: type of functions; 0 represents first-order functions, 1 represents higher-order functions
file_path: relative path to the scala files
code: the source code of higher-order functions calls
author: the author of higher-order functions calls
num_author: the number of commits to source code of the higher-order function definitions
num_commit: the number of authors of the higher-order function definitions