row_type,name,publication_date,organization,developer_countries,developer_sector,domain,count_kind,count_low,count_high,epoch_confidence,epoch_parameters_note,mixture_of_experts_note,unconfirmed_figure,not_a_neural_network,size_tier,training_compute_flop,neurons,scope,basis,first_model_at_or_above_low,first_model_at_or_above_high,source_ids,link,licence
model,Theseus,1950-07-02,Bell Laboratories,United States of America,Industry,Robotics,parameters,40,40,Confident,"The learned part is the maze configuration. There are 25 squares of the maze. The 16 squares to the left top corner have each one adjacent square down and one adjacent square up, for a total of 16*2 walls. We only need to count the 8 spare walls connecting the squares in the right side and the botto",,,,Below a fruit fly,40.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.technologyreview.com/2018/12/19/138508/mighty-mouse/,CC BY 4.0 (Epoch AI)
model,SNARC,1952-01-08,Harvard University,United States of America,Academia,Robotics,parameters,40,40,,"The link below seems to suggest the SNARC had 40 cells, each with a dial that acts as a configurable weight.  https://www.webofstories.com/play/marvin.minsky/137",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://en.wikipedia.org/wiki/Stochastic_neural_analog_reinforcement_calculator,CC BY 4.0 (Epoch AI)
model,Genetic algorithm,1954-07-02,Institute for Advanced Study,United States of America,Academia,"Mathematics,Biology",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007/BF01556771,CC BY 4.0 (Epoch AI)
model,Self Organizing System,1955-03-01,Massachusetts Institute of Technology (MIT),United States of America,Academia,Other,parameters,225,225,,Figure 4 contains the learnt weight matrix,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/1455292.1455309,CC BY 4.0 (Epoch AI)
model,Sequence-based pattern recognition,1955-03-01,Massachusetts Institute of Technology (MIT),United States of America,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/1455292.1455310,CC BY 4.0 (Epoch AI)
model,Perceptron Mark I,1957-01-01,"Cornell Aeronautical Laboratory,Cornell University",United States of America,Academia,Other,parameters,1000,1000,Likely,"""Figure 4.8 Illustration of the Mark 1 perceptron hardware. The photograph on the left shows how the inputs were obtained using a simple camera system in which an input scene, in this case a printed character, was illuminated by powerful lights, and an image focussed onto a 20 × 20 array of cadmium ",,,,Below a fruit fly,694894.9377361819,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blogs.umass.edu/brain-wars/files/2016/03/rosenblatt-1957.pdf,CC BY 4.0 (Epoch AI)
model,Pandemonium (morse),1959-02-01,Massachusetts Institute of Technology (MIT),United States of America,Academia,Language,parameters,,,Speculative,"The paper mentions 11 function types. Unclear how many times they are called (number of ""demons"" in their Pandemonium implementation).",,,,,600000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aitopics.org/doc/classics:504E1BAC/,CC BY 4.0 (Epoch AI)
model,Samuel Neural Checkers,1959-07-01,IBM,United States of America,Industry,Games,parameters,16,16,Likely,"""with 16 terms for generalization learning""  ""Mention has been made several times of the procedure for replacing terms in the scoring polynomial. The program, as it is currently running, contains 38 different terms (in addition to the piece-advantage term), 16 of these being included in the scoring ",,,,Below a fruit fly,428400000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/abstract/document/5392560,CC BY 4.0 (Epoch AI)
model,Pattern recognition and reading by machine,1959-12-01,Sandia Corporation,United States of America,Industry,Vision,parameters,2625,2625,,A two bit state is recorded for each of the 75 cell pairs and each of the 25+10 characters recognized.,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/1460299.1460326,CC BY 4.0 (Epoch AI)
model,Perceptron (1960),1960-03-30,Cornell Aeronautical Laboratory,United States of America,Academia,Vision,parameters,1000,1000,Speculative,""" The first program was designed to handle up to 1000 A units, and a 72 by 72 sensory mosaic. It was found that this large sensory system presented stimuli with a fineness of grain considerably better than the limits of discrimination of a thousand-unit percep- tron, and at the same time, required a",,yes,,Below a fruit fly,720000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Perceptron-Simulation-Experiments-Rosenblatt/ae76ce1ba27ac29addce4aab93b927e9bc7f7c67,CC BY 4.0 (Epoch AI)
model,ADALINE,1960-06-30,Stanford University,United States of America,Academia,Vision,parameters,17,17,Confident,"""The machine's total experience is stored in the values of the weights a0,...,a16""",,,,Below a fruit fly,6600.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf,CC BY 4.0 (Epoch AI)
model,Linear Decision Functions,1962-06-01,Bell Laboratories,United States of America,Industry,Mathematics,parameters,,,Speculative,,,,,,1559250.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/4066882?denied=,CC BY 4.0 (Epoch AI)
model,MADALINE I,1962-07-01,Stanford University,United States of America,Academia,Other,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.proquest.com/openview/7898314db50a218b58052ac91e3bde1e/1?,CC BY 4.0 (Epoch AI)
model,Print Recognition Logic,1963-01-01,IBM,United States of America,Industry,Vision,parameters,,,Speculative,,,,,,22500000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/5392331,CC BY 4.0 (Epoch AI)
model,Heuristic Reinforcement Learning,1965-10-01,Purdue University,United States of America,Academia,Robotics,parameters,,,Speculative,,,,,,1080000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/1098193,CC BY 4.0 (Epoch AI)
model,LTE speaker verification system,1966-11-01,IBM,United States of America,Industry,Speech,parameters,2061,2061,Likely,"2 connected systems, 1st level LTE and 2nd level LTE. 1st Level: 1810 parameters (""Thus, every 20 msec after the beginning of the utterance, the 15 filter amplitudes were each represented by a 12-bit code, resulting in a 180-bit time sample of the spectrum for that interval. Each time sample was fed",,,,Below a fruit fly,105917060.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://pubs.aip.org/asa/jasa/article-abstract/40/5/966/754180/Experimental-Studies-in-Speaker-Verification-Using?redirectedFrom=fulltext,CC BY 4.0 (Epoch AI)
model,Boxes (pole),1968-07-01,University of Edinburgh,United Kingdom,Academia,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.474.2430,CC BY 4.0 (Epoch AI)
model,GLEE,1968-07-01,University of Edinburgh,United Kingdom,Academia,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.474.2430,CC BY 4.0 (Epoch AI)
model,Decision tree adaline,1969-05-01,Tokyo Medical and Dental University,Japan,Academia,Medicine,parameters,2450,2450,Confident,5 adaline were trained on binary decisions (p. 1) Each adaline had up to 490 input weights (“meshes”) Total parameters = 5*490=2450 ,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://pubmed.ncbi.nlm.nih.gov/5820353/,CC BY 4.0 (Epoch AI)
model,Piecewise linear model,1973-11-01,University of Kansas,United States of America,Academia,Vision,parameters,357,357,Confident,"16 input features + bias = 17 input features 7*6/2 = 21 Hyperplanes 17*21 = 357 parameters ""For the multicategory problem involving NR categories, a total of NR(NR - 1)/2 hyperplanes are used to partition the pattern space."" ""The input variables to the classifier consisted of the mean variance of th",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/4309314,CC BY 4.0 (Epoch AI)
model,Cognitron,1975-09-01,Biological Cybernetics,Japan,Industry,Other,parameters,21600,21600,Confident,"4 layers, 288 neurons per layer, weights connect each neuron to only 25 neurons in the previous layer. Only 3 layers with learnable weights, the first layer is just an input representation (see Fig 5) 3*288*25 parameters",,,,Below a fruit fly,5184000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007%2FBF00342633,CC BY 4.0 (Epoch AI)
model,Transfer Learning,1976-07-01,University of Zagreb,Croatia,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.informatica.si/index.php/informatica/article/view/2828,CC BY 4.0 (Epoch AI)
model,Neocognitron,1980-04-01,NHK Broadcasting Science Research Laboratories,Japan,Industry,Vision,parameters,1140576,1140576,Confident,"""The synaptic connections from S-layers to C-layers are fixed and unmodifiable. [...] The numbers of excitatory cells in these seven layers are: 16x16 in U0, 16x16x24 in Us1, 10x10x 24 in Uc1, 8x8x24 in Us2, 6x 6x 24 in Uc2, 2x2x24 in Us3, and 24 in Uc3  [...]  the number of input synapses to each S",,,,Below a fruit fly,273738240.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007/BF00344251,CC BY 4.0 (Epoch AI)
model,Kohonen network,1981-07-25,Helsinki University of Technology,Finland,Academia,Mathematics,parameters,4096,4096,,"The input vectors are 3D. I could not find the grid size, but from the images it looks 8x8. So the network was 8x8x3 parameters.",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007/BF00337288,CC BY 4.0 (Epoch AI)
model,Hopfield network,1982-04-01,California Institute of Technology,United States of America,Academia,Other,parameters,9900,9900,,"My understanding is that the biggest Hopfield networks they studied had N=100 units.   Each unit has 99 synapses Tij from each other unit, for a total of 100*99 parameters",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.pnas.org/doi/10.1073/pnas.79.8.2554,CC BY 4.0 (Epoch AI)
model,ASE+ACE,1983-09-01,University of Massachusetts Amherst,United States of America,Academia,Robotics,parameters,324,324,Likely,"The system consists of two parts: ACE and ASE, each with 162 weights (=324 parameters). Found in Figures 2 and 3.",,,,Below a fruit fly,324000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6313077,CC BY 4.0 (Epoch AI)
model,Hierarchical Cognitron,1984-04-01,NHK Broadcasting Science Research Laboratories,Japan,Industry,Other,parameters,9315,9315,Speculative,"Parameters 5*5*9*3*3 + 3*3*9*3*3*9 + 9*3*3*9 = 9315 ""The numbers of excitatory cells in these four layers were: 7x7 in U0, 5x5 in  U1, 3x3 in U2, and 9 in U3"" ""Each feature-extracting cell in layer U1 receives excitatory modifiable afferent connections from 3x3 cells in layer U0"" ""On the other hand,",,yes,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007/BF00337157,CC BY 4.0 (Epoch AI)
model,Error Propagation,1986-01-03,"University of California San Diego,Carnegie Mellon University (CMU)",United States of America,Academia,Other,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://stanford.edu/~jlmcc/papers/PDP/Volume%201/Chap8_PDP86.pdf,CC BY 4.0 (Epoch AI)
model,Distributed representation NN,1986-08-15,Carnegie Mellon University (CMU),United States of America,Academia,Other,parameters,432,432,Confident,"Parameters: 24*6 + 12*6 + 12*12 + 12*6 =432 ""Figure 5: The activity levels in a five-layer network after it has learned. The bottom layer has 24 input units on the left for representing person 1 and 12 units on the right for representing the relationship. The white squares inside these two groups sh",yes,,,Below a fruit fly,388800000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.toronto.edu/~hinton/absps/families.pdf,CC BY 4.0 (Epoch AI)
model,MLP with back-propagation,1986-10-01,"University of California San Diego,Carnegie Mellon University (CMU)",United States of America,Academia,Mathematics,parameters,720,720,Confident,Architecture in Figure 3:   24+12 input -> 6 + 6 hidden -> 12 hidden -> 6 hidden -> 24 output  Parameters: 6*24+6*12+12*12+6*12+24*12=720,,,,Below a fruit fly,673920000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Learning-representations-by-back-propagating-errors-Rumelhart-Hinton/052b1d8ce63b07fec3de9dbb583772d860b7c769,CC BY 4.0 (Epoch AI)
model,Optimized Multi-Scale Edge Detection,1986-11-01,Massachusetts Institute of Technology (MIT),United States of America,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4767851,CC BY 4.0 (Epoch AI)
model,NetTalk (dictionary),1987-06-06,Princeton University,United States of America,Academia,Speech,parameters,18629,18629,Confident,"""The connections in the network are specified by a total of 18629 weight parameters (including a variable threshold for each unit)""",,,,Below a fruit fly,27664065000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=03A3D3EDF0BAF35405ABCF083411B55E?doi=10.1.1.154.7012&rep=rep1&type=pdf,CC BY 4.0 (Epoch AI)
model,NetTalk (transcription),1987-06-06,Princeton University,United States of America,Academia,Speech,parameters,18629,18629,Confident,"""The connections in the network are specified by a total of 18629 weight parameters (including a variable threshold for each unit)""",,,,Below a fruit fly,28328002560.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=03A3D3EDF0BAF35405ABCF083411B55E?doi=10.1.1.154.7012&rep=rep1&type=pdf,CC BY 4.0 (Epoch AI)
model,Translation-invariant MLP,1987-06-15,Carnegie Mellon University (CMU),United States of America,Academia,,parameters,816,816,Confident,Network: 12-60-6-16 Weights: 6*60+60*6+6*16=816 Layer 2 was only sparsely connected to input layer,yes,,,Below a fruit fly,18032947200.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.toronto.edu/~hinton/absps/parle.pdf,CC BY 4.0 (Epoch AI)
model,Latent semantic analysis,1988-04-05,"University of Chicago,Bell Laboratories,University of Western Ontario",Canada; United States of America,Mixed or other,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://asistdl.onlinelibrary.wiley.com/doi/10.1002/%28SICI%291097-4571%28199009%2941%3A6%3C391%3A%3AAID-ASI1%3E3.0.CO%3B2-9

https://www.cs.csustan.edu/~mmartin/LDS/Deerwester-et-al.pdf",CC BY 4.0 (Epoch AI)
model,MLN-ASR,1988-08-01,McGill University,Canada,Academia,Speech,parameters,10000,10000,Confident,"“For an MLN of about 10,000 links, the time was 115 CPU msecs for the recognition of a spoken letter and 317 msecs for the learning of a spoken letter on the SUN 4/280. A 20% reduction was obtained on the VAX 8650”",,,,Below a fruit fly,296425000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aaai.org/papers/00734-aaai88-130-data-driven-execution-of-multi-layered-networks-for-automatic-speech-recognition/,CC BY 4.0 (Epoch AI)
model,MLP baggage detector,1989-01-01,Science Applications International Corporation / SAIC,United States of America,Industry,Vision,parameters,,,Confident,"3 layer network, input layer (<20), hidden layer, output layer (3)",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Detection-of-explosives-in-checked-airline-baggage-Shea-Lin/71da4057401f459bc079696a029aee45a0a89728,CC BY 4.0 (Epoch AI)
model,Q-learning,1989-01-01,University of London,United Kingdom,Academia,"Robotics,Games",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://www.cs.rhul.ac.uk/~chrisw/thesis.html,CC BY 4.0 (Epoch AI)
model,Invariant image recognition,1989-06-18,Complutense University of Madrid,Spain,Academia,Vision,parameters,,,Confident,,,,,,27000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/118669,CC BY 4.0 (Epoch AI)
model,Truck backer-upper,1989-06-18,Stanford University,United States of America,Academia,Driving,parameters,805,805,Confident,6*25+25+8*45+45*6=805 (see Figure 6),,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/118723,CC BY 4.0 (Epoch AI)
model,Handwritten digit recognition network,1989-11-27,AT&T,United States of America,Industry,Vision,parameters,2578,2578,Confident,"""In summary, the network has 4635 units, 98442 connections, and 2578 independent parameters.“",,,,Below a fruit fly,181440000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Handwritten-Digit-Recognition-with-a-Network-LeCun-Boser/86ab4cae682fbd49c5a5bedb630e5a40fa7529f6,CC BY 4.0 (Epoch AI)
model,Speaker-independent vowel classification,1989-11-27,University of Washington,United States of America,Academia,Speech,parameters,3040,3040,Confident,"“The MLP consisted of 64 inputs (the DFf coefficients. each nonnalized between zero and one), a single hidden layer of 40 units, and 12 output units;”",,,,Below a fruit fly,7485696000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Performance-Comparisons-Between-Backpropagation-and-Atlas-Cole/e42d2b89fcb4a1a3dfa63408f424f76975ed1e1b,CC BY 4.0 (Epoch AI)
model,Zip CNN,1989-12-01,"AT&T,Bell Laboratories",United States of America,Industry,Vision,parameters,9760,9760,Confident,"""In summary, the network has 1256 units, 64,660 connections, and 9760 independent parameters""",,,,Below a fruit fly,1496338054440.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/6795724,CC BY 4.0 (Epoch AI)
model,NETtalk reimplementation,1990-06-01,Oregon State University,United States of America,Academia,Speech,parameters,27480,27480,Confident,"203*120+120*26=27480 “203 input units, 120 hidden units, and 26 output units”",,,,Below a fruit fly,35811936000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.sciencedirect.com/science/article/abs/pii/B9781558601413500079,CC BY 4.0 (Epoch AI)
model,Bankruptcy-NN,1990-06-17,,,Not stated,Other,parameters,36,36,Confident,"""The input layer consisted of the five nodes, one for each of the ratios. The hidden layer consisted of 5 node.;. The output layer consisted of only one neuron"" 30 weights + 6 biases = 36",,,,Below a fruit fly,3059337600.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/A-neural-network-model-for-bankruptcy-prediction-Odom-Sharda/ead9fa02902850a7418fb5ba720f3d9d8ab2f88b,CC BY 4.0 (Epoch AI)
model,ISR network,1990-10-01,Stanford University,United States of America,Academia,Vision,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://papers.nips.cc/paper_files/paper/1990/hash/e46de7e1bcaaced9a54f1e9d0d2f800d-Abstract.html,CC BY 4.0 (Epoch AI)
model,SexNet classification,1990-10-01,,,Not stated,Vision,parameters,1640,1640,Likely,Largest classification model: 40*40 + 40=1640 (Figure 2),,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/SEXNET%3A-A-Neural-Network-Identifies-Sex-From-Human-Golomb-Lawrence/cbf90aa78fea0c8a1028705d92bc4bc7808ddeeb,CC BY 4.0 (Epoch AI)
model,SexNet compression,1990-10-01,,,Not stated,Vision,parameters,72940,72940,Confident,900*40*2+40+900=72940 “Images sampled at 30x30 were compressed using a 900x40x900 fully connected back-propagation network”,,,,Below a fruit fly,78775200000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/SEXNET%3A-A-Neural-Network-Identifies-Sex-From-Human-Golomb-Lawrence/cbf90aa78fea0c8a1028705d92bc4bc7808ddeeb,CC BY 4.0 (Epoch AI)
model,RAAM,1990-11-01,,,Not stated,Other,parameters,1536,1536,Confident,"Largest model: ""A 48-16-48 RAAM learned to construct representations "" Parameters 48*16*2 = 1536",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.sciencedirect.com/science/article/abs/pii/000437029090005K?via%3Dihub,CC BY 4.0 (Epoch AI)
model,Weight Decay,1991-12-02,,,Not stated,Speech,parameters,8386,8386,Confident," 7*26*40+40+40*26+26=8386 ""The network had 7 x 26 input units, 40 hidden units and 26 output units""",,,,Below a fruit fly,75474000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/A-Simple-Weight-Decay-Can-Improve-Generalization-Krogh-Hertz/48e1de7d085808004d5f0493d486669a3d2930b5,CC BY 4.0 (Epoch AI)
model,TD-Gammon,1992-05-01,IBM,United States of America,Industry,Games,parameters,25000,25000,Speculative,"""The best performance was obtained with a network containing 80 hidden units and over 25,000 weights.""",,yes,,Below a fruit fly,18232157622832.703,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://papers.nips.cc/paper/1991/file/68ce199ec2c5517597ce0a4d89620f55-Paper.pdf,CC BY 4.0 (Epoch AI)
model,Golem,1992-10-01,Alan Turing Institute,United Kingdom,Mixed or other,Biology,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Protein-secondary-structure-prediction-using-Muggleton-King/9f744e48091a24b569435d070920e60db45f4fdc,CC BY 4.0 (Epoch AI)
model,Cancer drug mechanism prediction,1992-10-16,National Cancer Institute,United States of America,Mixed or other,Medicine,parameters,594,594,Likely,"“The network shown has 60 input PEs, one for each cell line, and 6 output PEs“ “Neural networks with three to nine hidden layer PEs used” 9*60 + 6*9 = 594",,,,Below a fruit fly,53460000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://pubmed.ncbi.nlm.nih.gov/1411538/,CC BY 4.0 (Epoch AI)
model,Boosting,1992-11-30,Bell Laboratories,United States of America,Industry,Vision,parameters,2578,2578,Likely,"“The network has 4645 neurons, 2578 different weights, and 98442 connections.“",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Improving-Performance-in-Neural-Networks-Using-a-Drucker-Schapire/77b5185dafb9e5b884a677a32713e54c253a4e0b,CC BY 4.0 (Epoch AI)
model,IBM-5,1993-06-15,IBM,United States of America,Industry,Language,parameters,1658364,1658364,,"The model is initiallized with 2.44E+09 translation probabilities, which are progressively culled until 1,658,364 remain. There are other parameters in the models (eg the fertility probabilities that relate each word in the input to the number of words it will align to) but the parameter count is do",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/972470.972474,CC BY 4.0 (Epoch AI)
model,Siamese-TDNN,1993-08-01,Bell Laboratories,United States of America,Industry,Vision,parameters,744,744,Likely,"""The input is 8 by 200 units, the first convolutional layer is 6 by 192 units with each unit's receptive field covering 8 by 9 units of the input. The first averaging layer is 6 by 64 units, the second convolution layer is 4 by 57 with 6 by 8 receptive fields and the second averaging layer is 4 by 1",,,,Below a fruit fly,12869570138112.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Signature-Verification-Using-A-%22Siamese%22-Time-Delay-Bromley-Bentz/997dc5d9a058753f034422afe7bd0cc0b8ad808b,CC BY 4.0 (Epoch AI)
model,ANN Eye Tracker,1993-11-29,,,Not stated,Vision,parameters,5620,5620,Confident,"15*15*20+20+50*10*2+100=5620 Hidden layer is split, 15*15 image input, 2*50 output neurons (see Figure 2) Hidden size up to 20 neurons (""This architecture was used with varying numbers of hidden units in the single, divided, hidden layer; experiments with 10, 16 and 20 hidden units were performed. """,,,,Below a fruit fly,17534400000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Non-Intrusive-Gaze-Tracking-Using-Artificial-Neural-Baluja-Pomerleau/574c0cf98825bf09b0ab7bbfe9ba89cd6090745e,CC BY 4.0 (Epoch AI)
model,Ceramic-MLP,1994-01-07,Sapienza Università di Roma,Italy,Academia,Materials science,parameters,1888,1888,Likely,"Parameters: 100*16 + 16*16 + 16*2 = 1888 Architecture: ""The topology of the classifier was X-Y-Y-2, where X is the number of input components, Y is the number of neurons in each hidden layer and the number of neurons in the output layer is two, which is the number of classes. The two hidden layers w",,,,Below a fruit fly,4531200000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.sciencedirect.com/science/article/abs/pii/S0921883108605506,CC BY 4.0 (Epoch AI)
model,GroupLens,1994-10-22,Massachusetts Institute of Technology (MIT),United States of America,Academia,Recommendation,parameters,,,,"For each pair of users, the system computes the correlation between their scores in the articles they have rated.  Then to make the prediction of a score for a given article and user the system computes a weighted average taking into account the correlations with each other user, the average rating ",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/192844.192905,CC BY 4.0 (Epoch AI)
model,JPMAX,1994-12-02,,,Not stated,Vision,parameters,4446,4446,Speculative,"Inputs are 12x12 pixels (Figure 3). They first train the architecture in Figure 2 a), then freeze it and train the additional layer in Figure 3 b).  Figure 2 a): 2*(12*12*15) + 2*15 = 4,350 Figure 2 b): 2*(12*12*15) + 2*15 + 2*(15*3) + 2*3 = 4,446",,yes,,Below a fruit fly,80828280.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper_files/paper/1994/hash/4b0250793549726d5c1ea3906726ebfe-Abstract.html,CC BY 4.0 (Epoch AI)
model,Mixture of linear models,1994-12-02,,,Not stated,Vision,parameters,384000,384000,Likely,"“In the example we describe, 7000 training images are sufficient to fit 384,000 parameters“",,,,Below a fruit fly,453600000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Recognizing-Handwritten-Digits-Using-Mixtures-of-Hinton-Revow/9dea20c1e5bbb1f543ff08113ffde5380c679f1f,CC BY 4.0 (Epoch AI)
model,NeuroChess,1994-12-02,,,Not stated,Games,parameters,72251,72251,Speculative,"""Prior to learning an evaluation function, the model M (175 input, 165 hidden, and 175 output units)"" = 58,090 parameters ""NeuroChess then learns an evaluation network V (175 input units, 0 to 80 hidden units, and one output units)."" = 14,161 parameters Total: 58,090 + 14,161 = 72,251",,yes,,Below a fruit fly,858730812676.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Learning-to-Play-the-Game-of-Chess-Thrun/4bc7a6dcb9e0e6c7a26800532e2a00f5572eea47,CC BY 4.0 (Epoch AI)
model,Predictive Coding NN,1994-12-02,Technical University of Munich,Germany,Academia,Language,parameters,206910,206910,Confident,"5*80*430+430+430*80+80=206910 ""P has nk input units and k output units. n is called the ""time-window size"" ""Note that the time-window was quite small (n = 5)."" ""alphabet consisted of k = 80 possible characters"" ""P had 430 hidden units""",,,,Below a fruit fly,18621900000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper/1994/file/5705e1164a8394aace6018e27d20d237-Paper.pdf,CC BY 4.0 (Epoch AI)
model,Support Vector Machines,1995-09-01,"AT&T,Bell Laboratories",United States of America,Industry,Vision,parameters,100000000,100000000,,"Section 6.2.2: ""...polynomials of degree 4 (that have more than 10^8 parameters)..."" They used 4-degree polynomials for MNIST",,,yes,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007/BF00994018,CC BY 4.0 (Epoch AI)
model,LISSOM,1995-11-27,University of Texas at Austin,United States of America,Academia,Vision,parameters,432800,432800,Likely,"Total connections 32*32*20*20+20*20*48+20*20*10=432800 Input: 32*32, Lissom: 20*20, Output: 10 (Figure 1a), up to 48 lateral connections per Lissom neuron (Figure 1b) ",,,,Below a fruit fly,195544800000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Laterally-Interconnected-Self-Organizing-Maps-in-Choe-Sirosh/785f5facc76538ceba6f6b9e2d7b641d322e9854,CC BY 4.0 (Epoch AI)
model,MUSIC perceptron,1996-06-03,,,Not stated,Vision,parameters,13607,13607,Confident,230*55+56*15+16*6+7*3=13607 (Figure 2),,,,Below a fruit fly,881733600000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/549237,CC BY 4.0 (Epoch AI)
model,System 11,1996-06-18,Carnegie Mellon University (CMU),United States of America,Academia,Vision,parameters,6452,6452,Confident,System 11 is a combination of Network 1 and Network 2  Network 1 has 2095 connections and network 2 has 4357 connections (see table 1),,,,Below a fruit fly,25859616000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/655647,CC BY 4.0 (Epoch AI)
model,AdaBoost.M2 Digit Recognition,1996-07-03,AT&T,United States of America,Industry,Vision,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cseweb.ucsd.edu/~yfreund/papers/boostingexperiments.pdf,CC BY 4.0 (Epoch AI)
model,SOM-CNN,1997-01-31,,,Not stated,Vision,parameters,32015,32015,Confident,"SOM: 125 L1: 20*3*3*3=540 L2: 25*3*3*20*0.3=1350 L3: 25*5*6*40=30000 Total: 125+540+1350+30000=32,015 (see Table 1)",,,,Below a fruit fly,31431600000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Face-recognition%3A-a-convolutional-neural-network-Lawrence-Giles/86890c82b589e24007c56e1f40c5f928a0e04183,CC BY 4.0 (Epoch AI)
model,Deep Blue,1997-05-01,IBM,United States of America,Industry,Games,parameters,8000,8000,Speculative,"""The new chess chip had a completely redesigned evaluation function, going from around 6400 features to over 8000""",,yes,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.sciencedirect.com/science/article/pii/S0004370201001291,CC BY 4.0 (Epoch AI)
model,Bidirectional RNN,1997-11-01,Advanced Telecommunications Research Institute,Japan,Industry,Speech,parameters,13000,13000,,"Page 7: ""The structures of all networks are adjusted so that each of them has about the same number of parameters (approximately 13 000 here""",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/650093,CC BY 4.0 (Epoch AI)
model,LSTM,1997-11-15,Technical University of Munich,Germany,Academia,Language,parameters,10504,10504,Confident,Table 2  http://www.bioinf.jku.at/publications/older/2604.pdf,,,,Below a fruit fly,31512000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://direct.mit.edu/neco/article-abstract/9/8/1735/6109/Long-Short-Term-Memory?redirectedFrom=fulltext,CC BY 4.0 (Epoch AI)
model,LeNet-5,1998-11-01,AT&T,United States of America,Industry,Vision,parameters,60000,60000,Confident,"""[LeNet5] contains 390408 connections, but only 60000 trainable parameters because of the weight sharing""",,,,Below a fruit fly,2810937600000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf,CC BY 4.0 (Epoch AI)
model,LSTM with forget gates,1999-01-02,IDSIA,Switzerland,Academia,Language,parameters,276,276,,See Table 1,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/818041,CC BY 4.0 (Epoch AI)
model,RECONTRA-categorized,1999-06-02,,,Not stated,Language,parameters,66780,66780,Likely,6*50*140+140*140+140*37=66780 Table 1  ,,,,Below a fruit fly,8013600000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Text-to-Text-Machine-Translation-Using-the-RECONTRA-Casta%C3%B1o-Casacuberta/47ee046b416d9258952cb8f4b0e2b6e65f334fad,CC BY 4.0 (Epoch AI)
model,RECONTRA-uncategorized,1999-06-02,,,Not stated,Language,parameters,112000,112000,Likely,"8*61*160+160*160+160*52=112000 ""a network with 61 input units, 52 outputs, 160 hidden traits and 8 (4+I+3) delayed inputs"" Table 1 ",,,,Below a fruit fly,3864000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Text-to-Text-Machine-Translation-Using-the-RECONTRA-Casta%C3%B1o-Casacuberta/47ee046b416d9258952cb8f4b0e2b6e65f334fad,CC BY 4.0 (Epoch AI)
model,IBM Model 4,1999-07-02,"University of Southern California,IBM,University of Pennsylvania",United States of America,Mixed or other,Language,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://www-i6.informatik.rwth-aachen.de/publications/download/266/al-onaizan--1999.pdf,CC BY 4.0 (Epoch AI)
model,Neural LM,2000-11-28,University of Montreal / Université de Montréal,Canada,Academia,Language,parameters,6906980,6906980,Confident,"(30959*100) + (8*100*120) + (120*30959) = 6,906,980 ""This is obtained with a network with the direct architecture, 100 randomly initialized words features, 120 hidden units, and n = 8 words of context."" ""The Hansard corpus (Canadian parliament proceedings, French version) is a stream of about 34 mil",,,,Below a fruit fly,6339000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://papers.nips.cc/paper_files/paper/2000/file/728f206c2a01bf572b5940d7d9a8fa4c-Paper.pdf,CC BY 4.0 (Epoch AI)
model,PoE MNIST,2000-11-28,University College London (UCL),United Kingdom,Academia,Vision,parameters,3925310,3925310,Confident,"10 models, one for each digit. Largest models: 500 epochs, 500 hidden units (Table 2) ""The largest network was the best, even though each digit model contains 392,500 parameters trained on only 4,400 images"" ""the classification network had 30 inputs and therefore 300 weights and 10 output biases.""  ",,,,Below a fruit fly,51810000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper_files/paper/2000/file/1f1baa5b8edac74eb4eaa329f14a0361-Paper.pdf,CC BY 4.0 (Epoch AI)
model,Immediate trihead,2001-07-06,Brown University,United States of America,Academia,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.3115/1073012.1073029,CC BY 4.0 (Epoch AI)
model,Gradient Boosting Machine,2001-10-01,Stanford University,United States of America,Academia,Mathematics,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://projecteuclid.org/journals/annals-of-statistics/volume-29/issue-5/Greedy-function-approximation-A-gradient-boostingmachine/10.1214/aos/1013203451.full,CC BY 4.0 (Epoch AI)
model,Decision tree (classification),2001-12-08,"Mitsubishi Electric Research Labs,Compaq CRL",United States of America,Industry,Vision,parameters,12000,12000,Likely,"Iteratively learns decision tree features, where each feature has 2 parameters (threshold and parity). They learn 6000 features total: ""The complete face detection cascade has 38 stages with over 6000 features""  Additional weights (see Table 1) are a temporary variable of the optimization and not pa",,,,Below a fruit fly,63000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.cmu.edu/~efros/courses/LBMV07/Papers/viola-cvpr-01.pdf,CC BY 4.0 (Epoch AI)
model,Thumbs Up?,2002-05-28,"Cornell University,IBM",United States of America,Mixed or other,Language,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/cs/0205070,CC BY 4.0 (Epoch AI)
model,Statistical Shape Constellations,2003-01-01,California Institute of Technology,United States of America,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/content/pdf/10.1007/3-540-45054-8_2.pdf,CC BY 4.0 (Epoch AI)
model,LDA,2003-02-02,Stanford University,United States of America,Academia,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://jmlr.org/papers/volume3/blei03a/blei03a.pdf,CC BY 4.0 (Epoch AI)
model,NPLM (AP News),2003-03-15,University of Montreal / Université de Montréal,Canada,Academia,Language,parameters,11904264,11904264,Confident,"""The number of parameters is |V|(1 + nm + h) + h(1 + (n − 1)m) [...] For example, consider the following architecture used in the experiments on the AP (Associated Press) news data: the vocabulary size is |V| = 17,964, the number of hidden units is h = 60, the order of the model is n = 6, the number",,,,Below a fruit fly,1666869200000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/944919.944966,CC BY 4.0 (Epoch AI)
model,NPLM (Brown),2003-03-15,University of Montreal / Université de Montréal,Canada,Academia,Language,parameters,4124233,4124233,Confident,"""The number of parameters is |V|(1 + nm + h) + h(1 + (n − 1)m) [...] For example, consider the following architecture used in the experiments on the AP (Associated Press) news data: the vocabulary size is |V| = 17,964, the number of hidden units is h = 60, the order of the model is n = 6, the number",,,,Below a fruit fly,132076260000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/944919.944966,CC BY 4.0 (Epoch AI)
model,Invariant CNN,2004-06-27,New York University (NYU),United States of America,Academia,Vision,parameters,90575,90575,Confident,"""The network has a total of 90,575 trainable parameters.""",,,,Below a fruit fly,974230000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Learning-methods-for-generic-object-recognition-to-LeCun-Huang/f354310098e09c1e1dc88758fca36767fd9d084d,CC BY 4.0 (Epoch AI)
model,LMICA,2004-12-01,,,Not stated,Vision,parameters,4096000,4096000,Confident,"64*64*1000=4096000 ""100000 samples of natural scenes of 64 × 64 pixels were given as X"" ""LMICA was carried out in 1000 layers"" ",,,,Below a fruit fly,2782080000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Linear-Multilayer-Independent-Component-Analysis-Matsuda-Yamaguchi/7061b01572fbff2e223ce3abb59f397895b1ebf1,CC BY 4.0 (Epoch AI)
model,Hierarchical LM,2005-01-06,,,Not stated,Language,parameters,,,Confident,,,,,,115848000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Hierarchical-Probabilistic-Neural-Network-Language-Morin-Bengio/c19fbefdeead6a4154a22a9c8551a18b1530033a,CC BY 4.0 (Epoch AI)
model,Histograms of Oriented Gradients,2005-06-25,INRIA,France,Academia,Vision,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/1467360,CC BY 4.0 (Epoch AI)
model,RankNet,2005-08-07,"Microsoft Research,Microsoft",United States of America,Industry,Search,parameters,5711,5711,Confident,"Model is ""a two layer net with 10 hidden units"" Input is of size 569 ""In all, we use 569 features"" Parameters: 569*10 + 10 for hidden layer 10*1 + 1 for output layer",,,,Below a fruit fly,3482081588304.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/abs/10.1145/1102351.1102363,CC BY 4.0 (Epoch AI)
model,TFE SVM,2006-02-02,"Centre de Recherche en Automatique de Nancy (CRAN),CENPARMI",Canada; France,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://hal.archives-ouvertes.fr/hal-00018426/en,CC BY 4.0 (Epoch AI)
model,FAST,2006-05-07,University of Cambridge,United Kingdom,Academia,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/chapter/10.1007/11744023_34,CC BY 4.0 (Epoch AI)
model,Crazy Stone,2006-05-29,INRIA,France,Academia,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://inria.hal.science/inria-00116992v1,CC BY 4.0 (Epoch AI)
model,SVM-CNN,2006-06-17,New York University (NYU),United States of America,Academia,Vision,parameters,90857,90857,Confident,,,,,Below a fruit fly,745200000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Large-scale-Learning-with-SVM-and-Convolutional-for-Huang-LeCun/cf03fdf52dd6e4249cbbdbd0bffbbbe5ca389feb,CC BY 4.0 (Epoch AI)
model,Spatial Pyramid Matching,2006-06-17,"INRIA,University of Illinois Urbana-Champaign (UIUC),Ecole Normale Supèrieure",France; United States of America,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://inc.ucsd.edu/mplab/users/marni/Igert/Lazebnik_06.pdf,CC BY 4.0 (Epoch AI)
model,Deep Belief Nets,2006-07-18,"University of Toronto,National University of Singapore",Canada; Singapore,Academia,Vision,parameters,1600000,1600000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf,CC BY 4.0 (Epoch AI)
model,Dimensionality Reduction,2006-07-18,University of Toronto,Canada,Academia,Vision,parameters,3800000,3800000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.toronto.edu/~hinton/science.pdf,CC BY 4.0 (Epoch AI)
model,Local Binary Patterns for facial recognition,2006-12-01,"University of Oulu,IEEE",Finland; Multinational,Mixed or other,Vision,parameters,,,Unknown,"Shallowly investigated, couldn't find much. ",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.456.1094&rep=rep1&type=pdf,CC BY 4.0 (Epoch AI)
model,Greedy layer-wise DNN training,2006-12-04,University of Montreal / Université de Montréal,Canada,Academia,Other,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/2976456.2976476,CC BY 4.0 (Epoch AI)
model,λ-WASP,2007-06-01,UT Austin,United States of America,Academia,Language,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.aclweb.org/anthology/P07-1121/,CC BY 4.0 (Epoch AI)
model,KN-LM,2007-06-22,Google,United States of America,Industry,Language,parameters,21000000000,21000000000,Likely,Table 2,,,yes,Honeybee to mouse,7.7303808e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Large-Language-Models-in-Machine-Translation-Brants-Popat/ba786c46373892554b98df42df7af6f5da343c9d,CC BY 4.0 (Epoch AI)
model,SB-LM,2007-06-22,Google,United States of America,Industry,Language,parameters,300000000000,300000000000,Likely,Table 2,,,yes,Mouse to cat,1.4494464e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Large-Language-Models-in-Machine-Translation-Brants-Popat/ba786c46373892554b98df42df7af6f5da343c9d,CC BY 4.0 (Epoch AI)
model,BLSTM for handwriting (1),2007-09-23,"University of Bern,IDSIA,Technical University of Munich",Germany; Switzerland,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://people.idsia.ch//~juergen/icdar_2007.pdf,CC BY 4.0 (Epoch AI)
model,Enhanced Neighborhood-Based Filtering,2007-10-28,AT&T,United States of America,Industry,Recommendation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/abstract/document/4470228,CC BY 4.0 (Epoch AI)
model,BLSTM for handwriting (2),2007-12-03,"University of Bern,IDSIA,Technical University of Munich",Germany; Switzerland,Academia,Vision,parameters,100881,100881,,"For the raw input representation, there were 4 input units and a total of 100,881 weights",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper/2007/hash/4b0250793549726d5c1ea3906726ebfe-Abstract.html,CC BY 4.0 (Epoch AI)
model,Deep Multitask NLP Network,2008-07-05,NEC Laboratories,United States of America,Industry,Language,parameters,1500000,1500000,Speculative,"With a word vector size of 50 and a vocabulary size of 30,000, the embedding matrix has 1,500,000 parameters. There are also some small convolutional and dense layers with far fewer parameters.",,yes,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/1390156.1390177,CC BY 4.0 (Epoch AI)
model,Denoising Autoencoders,2008-07-05,University of Montreal / Université de Montréal,Canada,Academia,Other,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/1390156.1390294,CC BY 4.0 (Epoch AI)
model,Sparse digit recognition SVM,2008-11-19,University of Lubeck,Germany,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://pubmed.ncbi.nlm.nih.gov/19000969/,CC BY 4.0 (Epoch AI)
model,BigChaos 2008,2008-11-25,AT&T,United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.asc.ohio-state.edu/statistics/statgen/joul_aut2009/BigChaos.pdf,CC BY 4.0 (Epoch AI)
model,HLBL,2008-12-08,University of Toronto,Canada,Academia,Language,parameters,1846400,1846400,Confident,"""Except for where stated otherwise, the models used for the experiments used 100 dimensional feature vectors and a context size of 5."" ""The vocabulary size for this dataset is 17964."" Embedding: 17964 * 100 = 1796400 Context matrices: 5 * 100 * 100 = 50000 Unembedding: 0 (tied embedding “while the m",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/A-Scalable-Hierarchical-Distributed-Language-Model-Mnih-Hinton/a9fc84f8abe740cdc7ee82e69444d1d00dbe0ceb,CC BY 4.0 (Epoch AI)
model,GNN,2008-12-09,University of Siena,Italy,Academia,Other,parameters,30,30,Confident,5*5+5=30 (3 layer network with 5 hidden neurons),,,,Below a fruit fly,1614600000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/4700287,CC BY 4.0 (Epoch AI)
model,BP-DBN,2009-01-01,University of Toronto,Canada,Academia,Speech,parameters,18030592,18030592,Confident,"429*2048+2048*2048*4+2048*183=18030592  Table 2: largest model has 5 layers of 2048 into 183 softmax, 429 input",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.utoronto.ca/~gdahl/papers/dbnPhoneRec.pdf,CC BY 4.0 (Epoch AI)
model,RBM Image Classifier,2009-04-08,University of Toronto,Canada,Academia,Vision,parameters,80000000,80000000,Likely,"Best performing model (see Figure 3.1) had 10,000 hidden units in one hidden layer and 8000 visible units",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf,CC BY 4.0 (Epoch AI)
model,GPU DBNs,2009-06-15,Stanford University,United States of America,Academia,Other,parameters,100000000,100000000,Confident,"""For example, we are able to reduce the time required to learn a four-layer DBN with 100 million parameters from several weeks to around a single day.""",,,,Fruit fly to honeybee,1000000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/abs/10.1145/1553374.1553486?utm_campaign=The+Batch&utm_source=hs_email&utm_medium=email&_hsenc=p2ANqtz-95Z7-X4Dl-RJK6gYKvjyDIrYaGhBeqWoc0ldqyPEKni0Ip6UE7452hr-ygY52z00LBpYgM,CC BY 4.0 (Epoch AI)
model,BellKor 2008,2009-08-01,AT&T,United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cseweb.ucsd.edu/classes/fa17/cse291-b/reading/ProgressPrize2008_BellKor.pdf,CC BY 4.0 (Epoch AI)
model,BellKor 2009,2009-08-01,AT&T,United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www2.seas.gwu.edu/~simhaweb/champalg/cf/papers/KorenBellKor2009.pdf,CC BY 4.0 (Epoch AI)
model,BigChaos OptiBlend,2009-08-01,AT&T,United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.asc.ohio-state.edu/statistics/statgen/joul_aut2009/BigChaos.pdf,CC BY 4.0 (Epoch AI)
model,Pragmatic Theory solution (Netflix 2009),2009-08-01,Pragmatic Theory Inc.,Canada,Industry,Recommendation,parameters,,,Unknown,"This is an ensemble of many smaller models. Ideally, the number of parameters of all the sub-models should be added up and recorded here.",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.asc.ohio-state.edu/statistics/statgen/joul_aut2009/PragmaticTheory.pdf,CC BY 4.0 (Epoch AI)
model,MatrixFac for Recommenders,2009-08-07,"Yahoo Research,AT&T",United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/5197422,CC BY 4.0 (Epoch AI)
model,Two Stage Feature Extraction (MNIST),2009-09-01,New York University (NYU),United States of America,Academia,Vision,parameters,258800,258800,Confident,CNN: 32*5*5+32*64*5*5=52000 FC: 1024*200+200*10=206800 Total: 52000+206800=258800,,,,Below a fruit fly,20754000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/What-is-the-best-multi-stage-architecture-for-Jarrett-Kavukcuoglu/1f88427d7aa8225e47f946ac41a0667d7b69ac52,CC BY 4.0 (Epoch AI)
model,BellKor 2007,2009-09-21,AT&T,United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/The-BellKor-solution-to-the-Netflix-Prize-Bell-Koren/f4ebb542c752a0dc423f94fd121e2edb8f6275ba,CC BY 4.0 (Epoch AI)
model,LCNP LabelMe,2009-11-22,University of Bonn,Germany,Academia,Vision,parameters,13729792,13729792,Confident,"Locally connected: 128*128*4*4*4*7 + 64*64*8*4*4*4 + 32*32*16*4*4*8 + 16*16*32*4*4*16=13631488 Classification head: 16*16*32*12=98304 Total: 13631488+98304=13729792 ""five regular layers with the dimensions 256×256, 128×128, . . ., 16×16."" ""size of the receptive field to be 4 × 4 neurons""",,,,Below a fruit fly,3295150080000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/5357786,CC BY 4.0 (Epoch AI)
model,LCNP MNIST,2009-11-22,,,Not stated,Vision,parameters,11616256,11616256,Confident,"Locally connected: 128*128*4*4*4*5 + 64*64*8*4*4*4 + 32*32*16*4*4*8 + 16*16*32*4*4*16=11534336 Classification head: 16*16*32*10=81920 Total: 11534336+81920=11616256 ""five regular layers with the dimensions 256×256, 128×128, . . ., 16×16."" ""size of the receptive field to be 4 × 4 neurons""",,,,Below a fruit fly,4181852160000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/5357786,CC BY 4.0 (Epoch AI)
model,LCNP NORB,2009-11-22,,,Not stated,Vision,parameters,16818176,16818176,Confident,"Locally connected: 128*128*4*4*4*10 + 64*64*8*4*4*4 + 32*32*16*4*4*8 + 16*16*32*4*4*16=16777216 Classification head: 16*16*32*5=40960 Total: 16777216+40960=16818176 ""five regular layers with the dimensions 256×256, 128×128, . . ., 16×16."" ""size of the receptive field to be 4 × 4 neurons""",,,,Below a fruit fly,2452090060800000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/5357786,CC BY 4.0 (Epoch AI)
model,Super-vector coding,2010-01-01,"University of Illinois Urbana-Champaign (UIUC),NEC Laboratories,Rutgers University",United States of America,Mixed or other,Vision,parameters,1025,1025,Speculative,"Somewhat low confidence, but it seems like the number of learnable parameters is the size of the codebook, plus parameters in the SVM used for classification. Since it is a linear SVM, there will be one parameter per input feature, plus a single bias term.  So in total, 512 learnable codebook values",,yes,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://tongzhang-ml.org/papers/eccv10_supervect.pdf,CC BY 4.0 (Epoch AI)
model,Stacked Denoising Autoencoders,2010-01-03,"University of Montreal / Université de Montréal,University of Toronto",Canada,Academia,Other,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.jmlr.org/papers/v11/vincent10a.html,CC BY 4.0 (Epoch AI)
model,Feedforward NN,2010-05-13,University of Montreal / Université de Montréal,Canada,Academia,Vision,parameters,7082000,7082000,Confident,"pg250 of the paper, section 2.3:  ""We optimized feedforward neural networks with one to five hidden layers, with one thousand hidden units per layer""  Input is a flattened 32x32 image, which corresponds to an input vector of length 3072  Output is a number from 0-9, so 10 neurons  No. of params: 307",,,,Below a fruit fly,350000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.mlr.press/v9/glorot10a.html,CC BY 4.0 (Epoch AI)
model,iCCCP,2010-06-13,Massachusetts Institute of Technology (MIT),United States of America,Academia,Vision,parameters,,,Confident,,,,,,1080000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/5540096,CC BY 4.0 (Epoch AI)
model,ReLU (LFW),2010-06-15,University of Toronto,Canada,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/3104322.3104425,CC BY 4.0 (Epoch AI)
model,ReLU (NORB),2010-06-15,University of Toronto,Canada,Academia,Vision,parameters,16210006,16210006,,"""The stereo-pair images are subsampled from their original resolution of 108 × 108 × 2 to 32 × 32 × 2 to speed up experiments [...]  the architecture with the best results have 4000 units in the first layer and 2000 in the second [...] there are 58,320 test cases (9,720 cases per class) ""  So the ar",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/3104322.3104425,CC BY 4.0 (Epoch AI)
model,Pooling CNN (Caltech 101),2010-09-15,University of Bonn,Germany,Academia,Vision,parameters,294912,294912,Confident,C1: 3*16*16*16=12288 C3: 16*128*6*6=73728 F5: 128*4*4*102=208896 Total: 12288+73728+208896=294912 ,,,,Below a fruit fly,1221124128768000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Evaluation-of-Pooling-Operations-in-Convolutional-Scherer-M%C3%BCller/5d21006fa32ff69f6b0a646f26ce0db84f2f4d33,CC BY 4.0 (Epoch AI)
model,Pooling CNN (NORB),2010-09-15,University of Bonn,Germany,Academia,Vision,parameters,268664,268664,Confident,Figure 1 C1: 2*16*5*5=800 C3: 16*64*6*6=36864 F5: 64*6*6*100=230400 F6: 100*6=600 Total: 800+36864+230400+600=268664,,,,Below a fruit fly,1456277227200000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Evaluation-of-Pooling-Operations-in-Convolutional-Scherer-M%C3%BCller/5d21006fa32ff69f6b0a646f26ce0db84f2f4d33,CC BY 4.0 (Epoch AI)
model,RNN LM,2010-09-26,Johns Hopkins University,United States of America,Academia,Language,parameters,70265000,70265000,Speculative,"This database entry refers to the 3xRNN rows in Table 2 (static and dynamic likely use the same model ensemble, but allow the model weights to update once when testing the dynamic version).  I assume the 3xRNN represents interpolation between the three largest models shown explicitly (RNN 250/5, RNN",,yes,,Fruit fly to honeybee,5.396e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Recurrent-neural-network-based-language-model-Mikolov-Karafi%C3%A1t/9819b600a828a57e1cde047bbe710d3446b30da5,CC BY 4.0 (Epoch AI)
model,YouTube Video Recommendation System,2010-09-26,Google,United States of America,Industry,Recommendation,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/1864708.1864770,CC BY 4.0 (Epoch AI)
model,Deep rectifier networks,2011-04-13,University of Montreal / Université de Montréal,Canada,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.mlr.press/v15/glorot11a.html,CC BY 4.0 (Epoch AI)
model,Deep Autoencoders,2011-04-29,University of Toronto,Canada,Academia,Vision,parameters,139808256,139808256,Confident,"2*(3072*8192+8192*4096+4096*2048+2048*1024+1024*512+512*256+256*128+128*64+64*28)=139808256 ""n each autoencoder, the hidden layers halve in size until they reach the desired size, except that we use 28 instead of 32""",,,,Fruit fly to honeybee,3.672864e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Using-very-deep-autoencoders-for-content-based-Krizhevsky-Hinton/88080d28536f36588740737f3b7a1f6c1a409654,CC BY 4.0 (Epoch AI)
model,Cross-Lingual POS Tagger,2011-06-19,"Carnegie Mellon University (CMU),Google Research",United States of America,Mixed or other,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aclanthology.org/P11-1061/,CC BY 4.0 (Epoch AI)
model,Vector Space Model,2011-06-19,Stanford University,United States of America,Academia,Language,parameters,255000,255000,Confident,"""We build a fixed dictionary of the 5,000 most frequent tokens"" ""For all word vector models, we use 50-dimensional vectors"" Parameters: 5000*50 + 5000=255000 ",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.semanticscholar.org/paper/Learning-Word-Vectors-for-Sentiment-Analysis-Maas-Daly/1c61f9ef06fe74505775a833ff849185757199e7

https://ai.stanford.edu/~ang/papers/acl11-WordVectorsSentimentAnalysis.pdf",CC BY 4.0 (Epoch AI)
model,Recursive Neural Network,2011-06-28,Stanford University,United States of America,Academia,"Vision,Language",parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Parsing-Natural-Scenes-and-Natural-Language-with-Socher-Lin/9c0ddf74f87d154db88d79c640578c1610451eec,CC BY 4.0 (Epoch AI)
model,High Performance CNN (NORB),2011-07-16,"IDSIA,SUPSI",Switzerland,Academia,Vision,parameters,4878300,4878300,Likely,L1: 6*300*6*6=64800 L3: 500*600*4*4=4800000 L5: (108/4)*500=13500 Total: 64800+4800000+13500=4878300,,,,Below a fruit fly,2.57985e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://people.idsia.ch/~juergen/ijcai2011.pdf,CC BY 4.0 (Epoch AI)
model,CNN Committee (MNIST),2011-09-18,IDSIA,Switzerland,Academia,Vision,parameters,120620,120620,Likely,1st CNN layer: 20*4*4=320 2nd CNN layer: 20*40*9*9=64800 1st FC layer: 40*3*3*150=54000 2nd FC layer: 150*10=1500 Total: 320+64800+54000+1500=120620,,,,Below a fruit fly,5.2e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Convolutional-Neural-Network-Committees-for-Ciresan-Meier/260a7615bbffec052d67dffde5bcf9b4687b50ee,CC BY 4.0 (Epoch AI)
model,CNN Committee (NIST),2011-09-18,IDSIA,Switzerland,Academia,Vision,parameters,128420,128420,Likely,1st CNN layer: 20*4*4=320 2nd CNN layer: 20*40*9*9=64800 1st FC layer: 40*3*3*150=54000 2nd FC layer: 150*62=9300 Total : 320+64800+54000+9300=128420 ,,,,Below a fruit fly,2.6e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Convolutional-Neural-Network-Committees-for-Ciresan-Meier/260a7615bbffec052d67dffde5bcf9b4687b50ee,CC BY 4.0 (Epoch AI)
model,Adaptive Subgrad,2011-10-03,"Technion - Israel Institute of Technology,Google,University of California (UC) Berkeley",Israel; United States of America,Mixed or other,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/1953048.2021068,CC BY 4.0 (Epoch AI)
model,CNN committee (traffic sign),2011-10-03,IDSIA,Switzerland,Academia,Vision,parameters,1388800,1388800,Likely,Parameters L1: 3*3*3*100=2700 L3: 100*4*4*150=240000 L5: 150*3*3*250=337500 L7: 250*4*4*200=800000 L8: 200*43=8600 Total: 2700+240000+337500+800000+8600=1388800 ,,,,Below a fruit fly,991981425600000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/A-committee-of-neural-networks-for-traffic-sign-Ciresan-Meier/dd7f8b53e6802787179a961e766760cbbe2d5011,CC BY 4.0 (Epoch AI)
model,NLP from scratch,2011-11-08,"NEC Laboratories,Princeton University",United States of America,Mixed or other,Language,parameters,5000000,5000000,,"""The capacity of our network architectures lies mainly in the word lookup table, which contains 50 × 100,000 parameters to train. [...] most of the trainable parameters are located in the lookup tables.""",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.jmlr.org/papers/volume12/collobert11a/collobert11a.pdf,CC BY 4.0 (Epoch AI)
model,Dropout (CIFAR),2012-06-03,University of Toronto,Canada,Academia,Vision,parameters,,,Confident,,,,,,4268700000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1207.0580,CC BY 4.0 (Epoch AI)
model,Dropout (ImageNet),2012-06-03,University of Toronto,Canada,Academia,Vision,parameters,,,Confident,"""We achieved comparable performance of 48.6% error using a single neural network with five convolutional hidden layers interleaved with “max-pooling” layer followed by two globally connected layers and a final 1000-way softmax layer""",,,,,2.731968e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1207.0580,CC BY 4.0 (Epoch AI)
model,Dropout (MNIST),2012-06-03,University of Toronto,Canada,Academia,Vision,parameters,5594010,5594010,Confident,"We show results for 4 nets (784-800-800-10, 784-1200-1200-10, 784-2000-2000-10, 784-1200-1200-1200-10)  784*2000+2000*2000+10*2000+6010=5594010",,,,Below a fruit fly,6039370800000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1207.0580,CC BY 4.0 (Epoch AI)
model,Dropout (TIMIT),2012-06-03,University of Toronto,Canada,Academia,Speech,parameters,48840185,48840185,,The input to the net is 21 adjacent frames with an advance of 10ms per frame. The neural net has 4 fully-connected hidden layers of 4000 units per layer and 185 “softmax” output units that are subsequently merged into the 39 distinct classes used for the benchmark.,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1207.0580,CC BY 4.0 (Epoch AI)
model,Unsupervised High-level Feature Learner,2012-07-12,Google,United States of America,Industry,Vision,parameters,1000000000,1000000000,Likely,"""To answer this, we train a 9-layered locally connected sparse autoencoder with pooling and local contrast normalization on a large dataset of images (the model has 1 billion connections, the dataset has 10 million 200x200 pixel images downloaded from the Internet)""",yes,,,Honeybee to mouse,6e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1112.6209,CC BY 4.0 (Epoch AI)
model,Context-dependent RNN,2012-07-27,"Microsoft Research,Brno University of Technology",Czechia; United States of America,Mixed or other,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.microsoft.com/en-us/research/wp-content/uploads/2012/07/rnn_ctxt_TR.sav_.pdf,CC BY 4.0 (Epoch AI)
model,LSTM LM,2012-09-09,RWTH Aachen University,Germany,Academia,Language,parameters,102720000,102720000,Speculative,"Multiple models were trained, the largest on transcribed French podcast data. ""We trained an LSTM LM using 300 hidden nodes and 27 M running words of indomain training data"" ""Corpus sizes in number of running words; the vocabulary size of the Treebank corpus is 10 K, for Quaero French it is 170 K"" E",,yes,,Fruit fly to honeybee,1.66e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/LSTM-Neural-Networks-for-Language-Modeling-Sundermeyer-Schl%C3%BCter/f9a1b3850dfd837793743565a8af95973d395a4e,CC BY 4.0 (Epoch AI)
model,AlexNet,2012-09-30,University of Toronto,Canada,Academia,Vision,parameters,60000000,60000000,Confident,"""Our neural network architecture has 60 million parameters.""",,,,Fruit fly to honeybee,4.7e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html,CC BY 4.0 (Epoch AI)
model,RNN+LDA+KN5+cache,2012-12-01,"Microsoft,Brno University of Technology",Czechia; United States of America,Mixed or other,Language,parameters,9000000,9000000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/rnn_ctxt.pdf,CC BY 4.0 (Epoch AI)
model,Bayesian automated hyperparameter tuning,2012-12-02,"University of Toronto,University of Sherbrooke,Harvard University",Canada; United States of America,Academia,Other,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1206.2944,CC BY 4.0 (Epoch AI)
model,DNN EM segmentation,2012-12-03,"IDSIA,SUPSI",Switzerland,Academia,Vision,parameters,218896,218896,Confident,Table 1 shows architecture 1st layer CNN: 48*4*4=768 2nd layer CNN: 48*48*5*5=57600 3rd layer CNN: 48*48*4*4=36864 4th layer CNN: 48*48*4*4=36864 1st layer FC: 48*3*3*200=86400 2nd layer FC: 200*2=400 Total parameters: 768+57600+36864+36864+86400+400=218896,,,,Below a fruit fly,4.78e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Neural-Networks-Segment-Neuronal-Membranes-in-Ciresan-Giusti/09193e19b59fc8f05bee9d6efbfb1607ca5b6501,CC BY 4.0 (Epoch AI)
model,DistBelief Speech,2012-12-03,Google,United States of America,Industry,Speech,parameters,47185920,47185920,Speculative,"""We used a deep network with five layers: four hidden layer with sigmoidal activations and 2560 nodes each, and a softmax output layer with 8192 nodes."" ""The network was fully-connected layer-to-layer, for a total of approximately 42 million model parameters."" 2560*2560*4+2560*8192=47185920",,yes,,Below a fruit fly,3.114e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Large-Scale-Distributed-Deep-Networks-Dean-Corrado/3127190433230b3dc1abd0680bb58dced4bcd90e,CC BY 4.0 (Epoch AI)
model,DistBelief Vision,2012-12-03,Google,United States of America,Industry,Vision,parameters,1700000000,1700000000,Likely,"""we used Downpour SGD to train the 1.7 billion parameter image model""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Large-Scale-Distributed-Deep-Networks-Dean-Corrado/3127190433230b3dc1abd0680bb58dced4bcd90e,CC BY 4.0 (Epoch AI)
model,DistBelief NNLM,2013-01-16,Google,United States of America,Industry,Language,parameters,,,Likely,,,,,,2.612736e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1301.3781,CC BY 4.0 (Epoch AI)
model,PreTrans-3L-250H,2013-03-22,University of Toronto,Canada,Academia,Speech,parameters,43000000,43000000,,Table 1,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1303.5778,CC BY 4.0 (Epoch AI)
model,Selective Search,2013-04-02,"University of Trento,University of Amsterdam",Italy; Netherlands,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/article/10.1007/s11263-013-0620-5,CC BY 4.0 (Epoch AI)
model,Multilingual DNN,2013-05-26,Google,United States of America,Industry,Speech,parameters,206899200,206899200,Confident,"""The input for the DNN is eleven contiguous frames of 40-dimensional log-filterbank features. The DNN consists of four hidden layers each with 2560 nodes"" Network structure: 3 multilingual shared layers, 1 language specific hidden layer + output layer (Figure 2) Language specific layer output sizes:",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Multilingual-acoustic-models-using-distributed-deep-Heigold-Vanhoucke/a41b826d23957d6ad4e9e794d20a583a9b567c5d,CC BY 4.0 (Epoch AI)
model,ReLU-Speech,2013-05-26,"Google,University of Toronto,New York University (NYU)",Canada; United States of America,Mixed or other,Speech,parameters,101706240,101706240,Likely,"""The overall input dimensionality is 1040,"" ""All layers of our networks have 2560 hidden units "" ""used to generate 7969 context-dependent tied acoustic states"" Largest model: 12 hidden layers (Fig 4) Parameters: 1040*2560+12*2560*2560+2560*7969=101706240 ",,,,Fruit fly to honeybee,1.2773376e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/On-rectified-linear-units-for-speech-processing-Zeiler-Ranzato/64da1980714cfc130632c5b92b9d98c2f6763de6,CC BY 4.0 (Epoch AI)
model,Hierarchical Scene Labeling (Stanford Background),2013-08-01,New York University (NYU),United States of America,Academia,Vision,parameters,51609600,51609600,Confident,"Figure 1 details architecture, weights are shared between scales “The network is then applied to each 3-dimension input map Xs. This input is transformed into a 16-dimension feature map, using a bank of 16 filters, 10 connected to the Y channel, the 6 others connected to the U and V channels. The se",,,,Below a fruit fly,2.3774688e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/6338939,CC BY 4.0 (Epoch AI)
model,RCTM,2013-10-01,University of Oxford,United Kingdom,Academia,Language,parameters,,,Likely,,,,,,9331200000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Recurrent-Continuous-Translation-Models-Kalchbrenner-Blunsom/944a1cfd79dbfb6fef460360a0765ba790f4027a,CC BY 4.0 (Epoch AI)
model,RNTN,2013-10-01,Stanford University,United States of America,Academia,Language,parameters,,,Likely,,,,,,1.422e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Recursive-Deep-Models-for-Semantic-Compositionality-Socher-Perelygin/687bac2d3320083eb4530bf18bb8f8f721477600,CC BY 4.0 (Epoch AI)
model,Word2Vec (large),2013-10-16,Google,United States of America,Industry,Language,parameters,692000000,692000000,Likely,"We discarded from the vocabulary all words that occurred less than 5 times in the training data, which resulted in a vocabulary of size 692K",,,,Honeybee to mouse,3.888e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1310.4546,CC BY 4.0 (Epoch AI)
model,Word2Vec (small),2013-10-16,Google,United States of America,Industry,Language,parameters,207600000,207600000,,"""We discarded from the vocabulary all words that occurred less than 5 times in the training data, which resulted in a vocabulary of size 692K [...] Starting with the same news data as in the previous experiments, we first constructed the phrase based training corpus and then we trained several Skip-",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1310.4546,CC BY 4.0 (Epoch AI)
model,R-CNN (T-net),2013-11-11,University of California (UC) Berkeley,United States of America,Academia,Vision,parameters,69003872,69003872,,Computed from architecture description in Caffee  https://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/detection.ipynb,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1311.2524,CC BY 4.0 (Epoch AI)
model,Visualizing CNNs,2013-11-12,New York University (NYU),United States of America,Academia,Vision,parameters,,,Confident,,,,,,5.32e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1311.2901,CC BY 4.0 (Epoch AI)
model,DeViSE,2013-12-05,Google,United States of America,Industry,Vision,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/DeViSE%3A-A-Deep-Visual-Semantic-Embedding-Model-Frome-Corrado/4aa4069693bee00d1b0759ca3df35e59284e9845,CC BY 4.0 (Epoch AI)
model,TransE,2013-12-05,"Universite de Technologie de Compiègne – CNRS,Google",France; United States of America,Mixed or other,Language,parameters,942000000,942000000,Speculative,"Based on the TransE architecture, the authors give a formula for how the model size scales with the dimensionality of the dataset. The model scale is proportional to: k*(n_e+n_r) where k is the embeddings dimension, n_e is the number of entities, and n_r is the number of relationships.  They studied",,yes,,Honeybee to mouse,1.340928e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://papers.nips.cc/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html,CC BY 4.0 (Epoch AI)
model,RNN for 1B words,2013-12-11,Google,United States of America,Industry,Language,parameters,20000000000,20000000000,Speculative,20B from Table 1,,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1312.3005,CC BY 4.0 (Epoch AI)
model,Network in Network,2013-12-16,National University of Singapore,Singapore,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1312.4400,CC BY 4.0 (Epoch AI)
model,DQN,2013-12-19,DeepMind,United Kingdom,Industry,Games,parameters,836096,836096,Confident,"""The input to the neural network consists is an 84 × 84 × 4 image produced by φ. The first hidden layer convolves 16 8 × 8 filters with stride 4 with the input image and applies a rectifier nonlinearity [10, 18]. The second hidden layer convolves 32 4 × 4 filters with stride 2, again followed by a r",,,,Below a fruit fly,2846883840000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1312.5602,CC BY 4.0 (Epoch AI)
model,Image generation,2013-12-20,University of Amsterdam,Netherlands,Academia,Vision,parameters,784000,784000,Confident,"""We trained generative models (decoders) and corresponding encoders (a.k.a. recognition models) having 500 hidden units in case of MNIST""  784*500*2=784000 (Ignoring latent dimension)",,,,Below a fruit fly,475200000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1312.6114,CC BY 4.0 (Epoch AI)
model,OverFeat,2013-12-21,New York University (NYU),United States of America,Academia,Vision,parameters,144000000,144000000,Confident,144M (Table 4),,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1312.6229,CC BY 4.0 (Epoch AI)
model,GloVe (32B),2014-01-01,Stanford University,United States of America,Academia,Language,parameters,120000000,120000000,,400k vocab * 300 vector dimensions,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://nlp.stanford.edu/projects/glove/,CC BY 4.0 (Epoch AI)
model,GloVe (6B),2014-01-01,Stanford University,United States of America,Academia,Language,parameters,120000000,120000000,,400k vocab * 300 vector dimensions,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://nlp.stanford.edu/projects/glove/,CC BY 4.0 (Epoch AI)
model,HyperNEAT,2014-03-05,University of Texas at Austin,United States of America,Academia,Games,parameters,239712,239712,,"""The ANN consists of three layers (Fig. 3): a substrate layer inwhich information from the game screen (raw pixels, objects, ornoise) is given as input to the network; a processing layer whichadds a nonlinear internal representation; and a nonlinear outputlayer from which actions are read and convey",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://ieeexplore.ieee.org/abstract/document/6756960

https://www.cs.utexas.edu/~mhauskn/papers/atari.pdf",CC BY 4.0 (Epoch AI)
model,Paragraph Vector,2014-05-14,Google,United States of America,Industry,Language,parameters,32000000,32000000,Confident," 75000*400+5000*400=32000000 ""We learn the word vectors and paragraph vectors using 75,000 training documents"" ""In PV-DM, the learned vector representations have 400 dimensions for both words and documents"" Paragraph embedding of dimension number of paragraphs * embedding size Word embedding of dime",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Distributed-Representations-of-Sentences-and-Le-Mikolov/f3de86aeb442216a8391befcacb49e58b478f512,CC BY 4.0 (Epoch AI)
model,AdaRNN,2014-06-01,Beihang University,China,Academia,Language,parameters,13040,13040,Confident,"D=25 ""For recursive neural models, the dimension of word vector is set to 25, and f = tanh is used as the nonlinearity function. We employ 10 composition matrices in AdaRNN."" Composition matrices: ""W ∈ R D×2D is the composition matrix, and b is the bias vector."" C=10 ""We employ 10 composition matric",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Adaptive-Recursive-Neural-Network-for-Twitter-Dong-Wei/06e122f475a21d92dba137609c40f35690217475,CC BY 4.0 (Epoch AI)
model,Dropout: SVHN,2014-06-01,University of Toronto,Canada,Academia,Vision,parameters,47795232,47795232,Confident,"""The best architecture that we found uses three convolutional layers each followed by a max-pooling layer. The convolutional layers have 96, 128 and 256 filters respectively. Each convolutional layer has a 5 × 5 receptive field applied with a stride of 1 pixel. Each max pooling layer pools 3 × 3 reg",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://jmlr.org/papers/v15/srivastava14a.html,CC BY 4.0 (Epoch AI)
model,GRUs,2014-06-03,"University of Montreal / Université de Montréal,Jacobs University,University of Maine",Canada; Germany; United States of America,Academia,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1406.1078,CC BY 4.0 (Epoch AI)
model,Two-stream ConvNets for action recognition,2014-06-09,University of Oxford,United Kingdom,Academia,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1406.2199,CC BY 4.0 (Epoch AI)
model,GANs,2014-06-10,University of Montreal / Université de Montréal,Canada,Academia,Image generation,parameters,,,Speculative,The paper outlines the G-D framework but doesn't provide information about the structures of their generator and discriminator.,,,,,5.184e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1406.2661,CC BY 4.0 (Epoch AI)
model,SPPNet,2014-06-18,"Microsoft,Xi’an Jiaotong University,University of Science and Technology of China (USTC)",China; United States of America,Mixed or other,Vision,parameters,,,Confident,,,,,,3.411072e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1406.4729,CC BY 4.0 (Epoch AI)
model,Fragment embedding,2014-06-21,Stanford University,United States of America,Academia,Vision,parameters,144496000,144496000,Likely,"Model contains a word embedding. a matrix combining two word embeddings, and image embedding (built upon a pretrained RCNN image model. Word embedding: 400000 * 200 =80000000 (""Here, We is a d × 400, 000 matrix that encodes a 1-of-k vector into a d-dimensional word vector representation (we use d = ",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Fragment-Embeddings-for-Bidirectional-Image-Karpathy-Joulin/7f1b111f0bb703b0bd97aba505728a9b0d9b2a54,CC BY 4.0 (Epoch AI)
model,RNN-WER,2014-06-22,"DeepMind,University of Toronto",Canada; United Kingdom,Mixed or other,Speech,parameters,26500000,26500000,Likely,"""The network had five levels of bidirectional LSTM hidden layers, with 500 cells in each layer, giving a total of ∼ 26.5M weights.""",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.mlr.press/v32/graves14.html,CC BY 4.0 (Epoch AI)
model,DeepFace,2014-06-23,"Tel Aviv University,Facebook",Israel; United States of America,Mixed or other,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/6909616,CC BY 4.0 (Epoch AI)
model,Multiresolution CNN,2014-06-23,"Google,Stanford University",United States of America,Mixed or other,Video,parameters,126125568,126125568,,"""Using shorthand notation, the full [single frame] architecture is C(96, 11, 3)-N-P-C(256, 5, 1)-N-P-C(384, 3, 1)-C(384, 3, 1)-C(256, 3, 1)-P-FC(4096)-FC(4096), where C(d, f, s) indicates a convolutional layer with d filters of spatial size f ×f, applied to the input with stride s""  Two such single-",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/6909619,CC BY 4.0 (Epoch AI)
model,SmooCT,2014-07-01,University College London (UCL),United Kingdom,Academia,Games,parameters,,,Likely,,,,,,6.9e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Self-play-Monte-Carlo-tree-search-in-computer-poker-Heinrich-Silver/7b687599b4425aa959036071030e1212a3b359c7,CC BY 4.0 (Epoch AI)
model,ACF-WIDER,2014-07-15,Chinese Academy of Sciences,China,Academia,Vision,parameters,6144,6144,Confident,"The paper trains a decision tree ensemble. “we choose 2048 as the number of weak classifiers contained in the soft cascade. As each weak classifier is a depth-2 decision tree, it takes only two comparing operations to apply a weak classifier, which is quite fast.” Parameters: 2048*(2^2-1)=6144 ",,,,Below a fruit fly,76380000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1407.4023,CC BY 4.0 (Epoch AI)
model,NPD,2014-08-06,IEEE,Multinational,Industry,Vision,parameters,313856,313856,Confident,"""Our final detector contains 1,226 deep quadratic trees"" ""(depth of eight in this paper)"" Parameters of the trained decision trees: 1226*(2^8)=313856 ",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/A-Fast-and-Accurate-Unconstrained-Face-Detector-Liao-Jain/fb67880d99ca29125866087dad2934ccc221378a,CC BY 4.0 (Epoch AI)
model,RNNsearch-50*,2014-09-01,"Jacobs University Bremen,University of Montreal / Université de Montréal",Canada; Germany,Academia,Language,parameters,,,Confident,,,,,,1.5552e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.0473,CC BY 4.0 (Epoch AI)
model,VGG16,2014-09-04,University of Oxford,United Kingdom,Academia,Vision,parameters,138000000,138000000,Confident,Source: Table 2 https://arxiv.org/abs/1409.1556,,,,Fruit fly to honeybee,1.2291e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.1556,CC BY 4.0 (Epoch AI)
model,VGG19,2014-09-04,University of Oxford,United Kingdom,Academia,Vision,parameters,144000000,144000000,Likely,Source: Table 2 https://arxiv.org/abs/1409.1556,,,,Fruit fly to honeybee,1.1e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.1556,CC BY 4.0 (Epoch AI)
model,Seq2Seq LSTM,2014-09-10,Google,United States of America,Industry,Language,parameters,1920000000,1920000000,Confident,The resulting LSTM has 384M parameters of which 64M are pure recurrent connections (32M for the “encoder” LSTM and 32M for the “decoder” LSTM). The paper uses an ensemble of 5 LSTMs.,,,,Honeybee to mouse,5.6e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.3215,CC BY 4.0 (Epoch AI)
model,SPN-4+KN5,2014-09-14,"Singapore University of Technology & Design,DSO National Laboratories",Singapore,Mixed or other,Language,parameters,5000000,5000000,Likely,Estimate from table 2 of https://arxiv.org/abs/1609.07843  The authors of the linked paper draw on estimates from table 3 of https://arxiv.org/pdf/1508.06615.pdf,,,,Below a fruit fly,4.4e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.comp.nus.edu.sg/~skok/papers/is14.pdf,CC BY 4.0 (Epoch AI)
model,GoogLeNet / InceptionV1,2014-09-17,"Google,University of Michigan,University of North Carolina",United States of America,Mixed or other,Vision,parameters,6797700,6797700,Confident,Computed summing the parameters on table 1 of section 5,,,,Below a fruit fly,1.51e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.4842,CC BY 4.0 (Epoch AI)
model,Deeply-supervised nets,2014-09-18,Microsoft Research,United States of America,Industry,Vision,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.5185,CC BY 4.0 (Epoch AI)
model,Spatially-Sparse CNN,2014-09-23,University of Warwick,United Kingdom,Academia,Vision,parameters,,,Unknown,Parameter count not stated but is probably derivable from the paper.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1409.6070v1,CC BY 4.0 (Epoch AI)
model,LRCN,2014-11-07,"UT Austin,University of Massachusetts Lowell,University of California (UC) Berkeley",United States of America,Academia,Video,parameters,142552000,142552000,,"1st model: CaffeNet fc6 feature extractor (4096-length vectors) -> LSTM with 1024 hidden units  2nd model: CaffeNet fc6 feature extractor (4096-length vectors) -> 2 layer LSTM with 1000 hidden units  3rd mode: Like the second, but has encoder and decoder LSTMs (both with 2 layers)  AlexNet (close re",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1411.4389,CC BY 4.0 (Epoch AI)
model,SC-NLM,2014-11-10,University of Toronto,Canada,Academia,"Multimodal,Vision,Language",parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Unifying-Visual-Semantic-Embeddings-with-Multimodal-Kiros-Salakhutdinov/2e36ea91a3c8fbff92be2989325531b4002e2afc,CC BY 4.0 (Epoch AI)
model,Fully Convolutional Networks,2014-11-14,University of California (UC) Berkeley,United States of America,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1411.4038,CC BY 4.0 (Epoch AI)
model,Cascaded LNet-ANet,2014-11-28,Chinese University of Hong Kong (CUHK),Hong Kong,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1411.7766,CC BY 4.0 (Epoch AI)
model,TA-CNN,2014-11-29,Chinese University of Hong Kong (CUHK),Hong Kong,Academia,Vision,parameters,706048,706048,Likely,Architecture details in Figure 4 Conv1: 3*7*7*32=4704 Conv2: 32*5*5*48=38400Conv3: 48*3*3*64=27648 Conv4: 64*3*3*96=55296 Fc5: 5*2*96*500=480000 Fc6: 500*200=100000 Total: 4704+38400+27648+55296+480000+100000=706048 ,,,,Below a fruit fly,1.0854e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1412.0069,CC BY 4.0 (Epoch AI)
model,SNM-skip,2014-12-03,Google,United States of America,Industry,Language,parameters,62000000000,62000000000,Speculative,62B from Table 2,,yes,,Honeybee to mouse,2.97600000001e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1412.1454,CC BY 4.0 (Epoch AI)
model,Fractional Max-Pooling,2014-12-18,University of Warwick,United Kingdom,Academia,Vision,parameters,27000000,27000000,Likely,27M weights in largest CIFAR-100 model,,,,Below a fruit fly,1e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1412.6071v4,CC BY 4.0 (Epoch AI)
model,ADAM (CIFAR-10),2014-12-22,"University of Amsterdam,OpenAI,University of Toronto",Canada; Netherlands; United States of America,Mixed or other,Vision,parameters,2370000,2370000,Confident,"CIFAR-10 with c64-c64-c128-1000 architecture  ""Our CNN architecture has three alternating stages of 5x5 convolution filters and 3x3 max pooling with stride of 2 that are followed by a fully connected layer of 1000 rectified linear hidden units (ReLU’s). ""  ""Training cost over 45 epochs. CIFAR-10 wit",,,,Below a fruit fly,624979800000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1412.6980,CC BY 4.0 (Epoch AI)
model,VGG-Face,2015-01-01,University of Oxford,United Kingdom,Academia,Vision,parameters,138000000,138000000,Confident,Their largest model uses the VGG16 architecture,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Face-Recognition-Parkhi-Vedaldi/162ea969d1929ed180cc6de9f0bf116993ff6e06,CC BY 4.0 (Epoch AI)
model,"MSRA (C, PReLU)",2015-02-06,Microsoft Research,United States of America,Industry,Vision,parameters,87048800,87048800,Confident,"I used the architecture in table 3 I ignored biases, and assumed a SPP bin size of 256  3*7*7*96+96*3*3*384+384*3*3*384*5+384*3*3*768+768*3*3*768*5+768*3*3*896+896*3*3*896*5+896*(7*7+3*3+2*2+1)*4096+4096*4096+4096*1000=330581792    ",,,,Fruit fly to honeybee,2.397403008e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1502.01852,CC BY 4.0 (Epoch AI)
model,TRPO,2015-02-19,University of California (UC) Berkeley,United States of America,Academia,Games,parameters,33500,33500,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/1502.05477,CC BY 4.0 (Epoch AI)
model,DQN-2015,2015-02-25,Google,United States of America,Industry,Games,parameters,1693362,1693362,,"""The input to the neural network consists of an 84x84x4 image produced by the preprocess-ing mapw. The first hidden layer convolves 32 filters of 8x8 with stride 4 with theinput image and applies a rectifier nonlinearity. The second hidden layer con-volves 64 filters of 4x4 with stride 2, again foll",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/nature14236,CC BY 4.0 (Epoch AI)
model,genCNN + dyn eval,2015-03-17,"Chinese Academy of Sciences,Huawei Noah's Ark Lab,Dublin City University",China; Ireland,Mixed or other,Language,parameters,8000000,8000000,Speculative,8M according to https://arxiv.org/pdf/1508.06615,,yes,,Below a fruit fly,3.4153451e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aclanthology.org/P15-1151/,CC BY 4.0 (Epoch AI)
model,TC-DNN-BLSTM-DNN,2015-04-06,Carnegie Mellon University (CMU),United States of America,Academia,Speech,parameters,18413568,18413568,Confident,3*40 inputs - 2L DNN (2048) - BLSTM (128*2) - 2L DNN (2048) - 3431 outputs 3*40*2048 + 2048*2048 + 2*4*(2048+128)*128 + 256*2048 + 2048*2048 + 2048*3431=18413568 ,,,,Below a fruit fly,1.9410191999999997e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Recurrent-Neural-Networks-for-Acoustic-Chan-Lane/ae18f67c10a5d4da91f128ec7a6cf7c784122cd5,CC BY 4.0 (Epoch AI)
model,Fast R-CNN,2015-04-30,Microsoft Research,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1504.08083,CC BY 4.0 (Epoch AI)
model,U-Net,2015-05-18,University of Freiburg,Germany,Academia,Vision,parameters,37676160,37676160,Confident,Figure 1: 2*(1*3*3*64+64*3*3*64+64*3*3*128+128*3*3*128+128*3*3*256+256*3*3*256+256*3*3*512+512*3*3*512+512*3*3*1024+1024*3*3*1024)=37676160,,,,Below a fruit fly,5.0832252e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/U-Net%3A-Convolutional-Networks-for-Biomedical-Image-Ronneberger-Fischer/6364fdaa0a0eccd823a779fcdd489173f938e91a,CC BY 4.0 (Epoch AI)
model,Faster R-CNN,2015-06-04,Microsoft Research,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1506.01497,CC BY 4.0 (Epoch AI)
model,CFSS,2015-06-07,"SenseTime,Chinese University of Hong Kong (CUHK),Shenzhen Institute of Advanced Technology",China; Hong Kong,Mixed or other,Vision,parameters,17408,17408,Likely,"The paper trains two regression models, each using hardcoded SIFT / BRIEF features.  68 keypoints (given in the paper) and 128 dimensional features per keypoint (assumption based on SIFT features) 2*68*128=17408",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Face-alignment-by-coarse-to-fine-shape-searching-Zhu-Li/dc661f478c5a245a919c4b3faae8ea4f598b6e64,CC BY 4.0 (Epoch AI)
model,YOLO,2015-06-08,"University of Washington,Allen Institute for AI,Facebook AI Research",France; United States of America,Mixed or other,Vision,parameters,271684800,271684800,,Calculation based on figure 3 of the paper: 7 * 7 * 3 * 64 + 3 * 3 * 64 * 192 + 1 * 1 * 192 * 128 + 3 * 3 * 128 * 256 + 1 * 1 * 256 * 256 + 3 * 3 * 256 * 512 + 4 * (1 * 1 * 512 * 256 + 3 * 3 * 256 * 512) + 1 * 1 * 512 * 512 + 3 * 3 * 512 * 1024 + 2 * (1 * 1 * 1024 * 512 + 3 * 3 * 512 * 1024) + 4 * (,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1506.02640,CC BY 4.0 (Epoch AI)
model,BatchNorm,2015-06-15,Google,United States of America,Industry,Vision,parameters,13600000,13600000,Confident,"""The network contains 13.6 · 106 parameters""",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1502.03167,CC BY 4.0 (Epoch AI)
model,CompACT-Deep,2015-07-19,University of California San Diego,United States of America,Academia,Vision,parameters,,,Unknown,Uses combinations of finetuned VGG and AlexNet models,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Learning-Complexity-Aware-Cascades-for-Deep-Cai-Saberian/c3bd3b9782dc504ee4f2b8a12bd9c562a5c0d7ad,CC BY 4.0 (Epoch AI)
model,Deep CNN + COTS,2015-07-26,IEEE,Multinational,Industry,Vision,parameters,5006000,5006000,Confident,Taken from Table 1 of https://arxiv.org/abs/1508.01722 which uses the same architecture,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Face-Search-at-Scale%3A-80-Million-Gallery-Wang-Otto/be38934db99182c0e3100be58cd0299151b30625,CC BY 4.0 (Epoch AI)
model,DCNN,2015-08-07,"University of Maryland,Rutgers University",United States of America,Academia,Vision,parameters,5006000,5006000,Confident,Table 1,,,,Below a fruit fly,4.8098691e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1508.01722,CC BY 4.0 (Epoch AI)
model,BPE,2015-08-31,University of Edinburgh,United Kingdom,Academia,Language,parameters,,,,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1508.07909,CC BY 4.0 (Epoch AI)
model,Deep Deterministic Policy Gradients,2015-09-09,Google DeepMind,United States of America,Industry,Robotics,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1509.02971,CC BY 4.0 (Epoch AI)
model,AlphaGo Fan,2015-10-01,DeepMind,United Kingdom,Industry,Games,parameters,8209984,8209984,Likely,"The input to the policy network is a 19 × 19 × 48 image stack consisting of 48 feature planes. The first hidden layer zero pads the input into a 23 × 23 image, then convolves k filters of kernel size 5 × 5 with stride 1 with the input image and applies a rectifier nonlinearity. Each of the subsequen",,,,Below a fruit fly,3.8e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/nature16961,CC BY 4.0 (Epoch AI)
model,SAF R-CNN,2015-10-28,"Beijing Institute of Technology,Sun Yat-sen University,Panasonic R&D,National University of Singapore",China; Singapore,Mixed or other,Vision,parameters,138000000,138000000,Likely,"Taken from VGG16 base model, ignoring minor architectural changes",,,,Fruit fly to honeybee,1.2311081250000001e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1510.08160,CC BY 4.0 (Epoch AI)
model,3DDFA,2015-11-23,"Chinese Academy of Sciences,Michigan State University",China; United States of America,Academia,Vision,parameters,5355584,5355584,Confident,Parameters based on Figure 2: CNN layers: 4*4*16*6+4*4*32*16=9728 Local CNN: 3*3*48*32*11*11+3*3*64*49*9*9=3958848 FC: 9*9*64*256+234*256=1387008 Total: 9728+3958848+1387008=5355584 ,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1511.07212,CC BY 4.0 (Epoch AI)
model,Multi-scale Dilated CNN,2015-11-23,"Princeton University,Intel Labs",United States of America,Mixed or other,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1511.07122,CC BY 4.0 (Epoch AI)
model,Netflix Recommender System,2015-12-01,Netflix,United States of America,Industry,Recommendation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/pdf/10.1145/2843948,CC BY 4.0 (Epoch AI)
model,Inception v3,2015-12-02,"Google,University College London (UCL)",United Kingdom; United States of America,Mixed or other,Vision,parameters,23626728,23626728,Likely,Table 3 from Xception paper,,,,Below a fruit fly,1e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1512.00567,CC BY 4.0 (Epoch AI)
model,SSD,2015-12-08,,,Not stated,Vision,parameters,,,Confident,Can be calculated from the VGG-16 paper and Figure 2,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/1512.02325,CC BY 4.0 (Epoch AI)
model,ResNet-101 (ImageNet),2015-12-10,Microsoft,United States of America,Industry,Vision,parameters,44500000,44500000,Confident,Taken from https://arxiv.org/abs/1605.07146,,,,Below a fruit fly,7.004e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1512.03385,CC BY 4.0 (Epoch AI)
model,ResNet-110 (CIFAR-10),2015-12-10,Microsoft,United States of America,Industry,Vision,parameters,1700000,1700000,,Table 6,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1512.03385,CC BY 4.0 (Epoch AI)
model,ResNet-152 (ImageNet),2015-12-10,Microsoft,United States of America,Industry,Vision,parameters,60200000,60200000,Confident,Taken from https://arxiv.org/abs/1605.07146,,,,Fruit fly to honeybee,1.041408e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1512.03385,CC BY 4.0 (Epoch AI)
model,Advantage Learning,2015-12-15,Google DeepMind,United States of America,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://arxiv.org/abs/1512.04860v1,CC BY 4.0 (Epoch AI)
model,"Variational (untied weights, MC) LSTM (Large)",2015-12-16,University of Cambridge,United Kingdom,Academia,Language,parameters,66000000,66000000,Confident,66M according to https://arxiv.org/pdf/1611.01462,,,,Fruit fly to honeybee,5886144000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1512.05287,CC BY 4.0 (Epoch AI)
model,AlphaGo Lee,2016-01-27,DeepMind,United Kingdom,Industry,Games,parameters,,,Speculative,,,,,,1.9e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/nature16961,CC BY 4.0 (Epoch AI)
model,A3C FF hs,2016-02-04,"Google,University of Montreal / Université de Montréal",Canada; United States of America,Mixed or other,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://arxiv.org/abs/1602.01783v2,CC BY 4.0 (Epoch AI)
model,Inception-ResNet-V2,2016-02-23,Google,United States of America,Industry,Vision,parameters,56000000,56000000,,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1602.07261,CC BY 4.0 (Epoch AI)
model,Inceptionv4,2016-02-23,Google,United States of America,Industry,Vision,parameters,43000000,43000000,,"""The folks from Google strike again with Inception-v4, 43M parameters.""  https://towardsdatascience.com/illustrated-10-cnn-architectures-95d78ace614d",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1602.07261,CC BY 4.0 (Epoch AI)
model,SqueezeNet,2016-02-24,"DeepScale,University of California (UC) Berkeley,Stanford University",United States of America,Academia,Vision,parameters,1200000,1200000,,"The paper says ""SqueezeNet achieves AlexNet-level accuracy on ImageNet with 50x fewer parameters.""  AlexNet has 60 million parameters.",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1602.07360,CC BY 4.0 (Epoch AI)
model,Double DQN,2016-03-02,Google DeepMind,United States of America,Industry,Games,parameters,1500000,1500000,Confident,"""approximately 1.5M parameters in total""",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ojs.aaai.org/index.php/AAAI/article/view/10295,CC BY 4.0 (Epoch AI)
model,Named Entity Recognition model,2016-03-04,Carnegie Mellon University (CMU),United States of America,Academia,Language,parameters,,,Confident,Architecture in Table 1,,,,,9.69408e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1603.01354,CC BY 4.0 (Epoch AI)
model,Template Adaptation,2016-03-12,University of Oxford,United Kingdom,Academia,Vision,parameters,138000000,138000000,Likely,Trains SVM on top of VGG-Face (VGG16 architecture). SVM parameters are not included in the estimate.,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Template-Adaptation-for-Face-Verification-and-Crosswhite-Byrne/42515cd370f5edf4ac6c8d4bc3f6e0d848f6b73a,CC BY 4.0 (Epoch AI)
model,Dueling DQN,2016-04-05,Google DeepMind,United States of America,Industry,Games,parameters,1700000,1700000,Confident,Same parameter count as DQN,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1511.06581,CC BY 4.0 (Epoch AI)
model,Gated HORNN (3rd order),2016-04-30,York University,Canada,Academia,Language,parameters,8970000,8970000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1605.00064,CC BY 4.0 (Epoch AI)
model,LRR-4X,2016-05-08,UC Irvine,United States of America,Academia,Vision,parameters,138000000,138000000,Likely,VGG16 with some added layers which are not specified in detail.,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Laplacian-Pyramid-Reconstruction-and-Refinement-for-Ghiasi-Fowlkes/2796c448023b78fd77f3a4b57966f257c5e654c2,CC BY 4.0 (Epoch AI)
model,PixelCNN,2016-06-16,Google DeepMind,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1606.05328,CC BY 4.0 (Epoch AI)
model,CMS-RCNN,2016-06-17,IEEE,Multinational,Industry,Vision,parameters,138000000,138000000,Likely,"Based on VGG16 architecture, actual parameter count is slightly larger since the model contains some small added layers which are not described in detail.",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/CMS-RCNN%3A-Contextual-Multi-Scale-Region-based-CNN-Zhu-Zheng/e8b2a98f87b7b2593b4a046464c1ec63bfd13b51,CC BY 4.0 (Epoch AI)
model,Segmental RNN,2016-06-20,"University of Edinburgh,Carnegie Mellon University (CMU),University of Washington",United Kingdom; United States of America,Academia,Speech,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1603.00223,CC BY 4.0 (Epoch AI)
model,R-FCN,2016-06-21,"Tsinghua University,Microsoft Research",China; United States of America,Mixed or other,Vision,parameters,,,Confident,,,,,,7.1935776e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1605.06409,CC BY 4.0 (Epoch AI)
model,CCL,2016-06-27,"SenseTime,Chinese University of Hong Kong (CUHK),Chinese Academy of Sciences",China; Hong Kong,Mixed or other,Vision,parameters,,,Confident,The paper trains multiple regressions but the total parameter count is unclear.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Unconstrained-Face-Alignment-via-Cascaded-Learning-Zhu-Li/4396563ee3ec167932071e7936111f6a8899e905,CC BY 4.0 (Epoch AI)
model,Character-enriched word2vec,2016-07-15,Facebook AI Research,France; United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1607.04606,CC BY 4.0 (Epoch AI)
model,SimpleNet,2016-08-22,"Sensifai,Islamic Azad University,Technicolor R&I,Institute for Research in Fundamental Sciences (IPM)",Belgium; France; Iran (Islamic Republic of),Mixed or other,Vision,parameters,5480000,5480000,Confident,SOTA CIFAR-10 model was 5.48m params,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1608.06037,CC BY 4.0 (Epoch AI)
model,DenseNet-264,2016-08-25,"Tsinghua University,Facebook AI Research,Cornell University",China; France; United States of America,Mixed or other,Vision,parameters,34000000,34000000,Confident,Figure 3,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1608.06993,CC BY 4.0 (Epoch AI)
model,LF-MMI,2016-09-08,"Johns Hopkins University,Cornell University",United States of America,Academia,Speech,parameters,16600000,16600000,Confident,Largest model: TDNN-A: 16.6 million parameters (Table 2),,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Purely-Sequence-Trained-Neural-Networks-for-ASR-on-Povey-Peddinti/6ce6a9a30cd69bd2842a4b581cf48c6815bdfdd8,CC BY 4.0 (Epoch AI)
model,MS-ensemble-speech-recognition,2016-09-12,Microsoft,United States of America,Industry,Speech,parameters,3172117056,3172117056,Likely,Large ensemble CNNs: 85000000 + 38000000 + 65000000 = 188000000 = 188M LSTMs: 40 / 140 input - 6*512 layers - 9000 / 27000 output 4*(140+512)*512 + 5*4*(512+512)*512 + 4*(512+27000)*27000=2983117056 LM: 1000*1000 = 1000000 + Embedding Total: 1000000+2983117056+188000000=3172117056 (underestimate bec,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/The-microsoft-2016-conversational-speech-system-Xiong-Droppo/ac94ef90be9b0c3bf744d6744e47b38855f9a4c7,CC BY 4.0 (Epoch AI)
model,WaveNet,2016-09-12,Google DeepMind,United States of America,Industry,Speech,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1609.03499,CC BY 4.0 (Epoch AI)
model,Youtube recommendation model,2016-09-15,Google,United States of America,Industry,Recommendation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.google/pubs/pub45530/,CC BY 4.0 (Epoch AI)
model,ResNet-1001,2016-09-17,Microsoft,United States of America,Industry,Vision,parameters,10200000,10200000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/chapter/10.1007/978-3-319-46493-0_38,CC BY 4.0 (Epoch AI)
model,ResNet-200,2016-09-17,Microsoft Research Asia,China,Industry,Vision,parameters,,,Speculative,,,,,,2.9741645e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/chapter/10.1007/978-3-319-46493-0_38,CC BY 4.0 (Epoch AI)
model,Wide Residual Network,2016-09-19,Université Paris-Est,France,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1605.07146,CC BY 4.0 (Epoch AI)
model,GNMT,2016-09-26,Google,United States of America,Industry,Language,parameters,278000000,278000000,Likely,Table 5 in 'Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer'  https://arxiv.org/abs/1701.06538,yes,,,Fruit fly to honeybee,6.620000000001e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1609.08144,CC BY 4.0 (Epoch AI)
model,Pointer Sentinel-LSTM (medium),2016-09-26,"MetaMind Inc,Salesforce",United States of America,Industry,Language,parameters,21000000,21000000,Confident,,,,,Below a fruit fly,7490000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1609.07843,CC BY 4.0 (Epoch AI)
model,Xception,2016-10-07,Google,United States of America,Industry,Vision,parameters,22855952,22855952,Confident,Table 3,,,,Below a fruit fly,4.36e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1610.02357,CC BY 4.0 (Epoch AI)
model,GAWWN,2016-10-08,"University of Michigan,Max Planck Institute for Informatics",Germany; United States of America,Academia,Vision,parameters,,,Confident,DCGAN-style generator/discriminator with up to 128×128 output Char-CNN-GRU encoder (~20–25M parameters) from [Reed et al. 2016a]. Dual-pathway generator/discriminator roughly doubles convolutional parameters.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Learning-What-and-Where-to-Draw-Reed-Akata/cad4ac0d2389a89cf1955dd4788278c1e8ac1af9,CC BY 4.0 (Epoch AI)
model,SPIDER2,2016-10-28,"Griffith University,University of Iowa,Dezhou University",Australia; China; United States of America,Academia,Biology,parameters,409536,409536,Likely,"Three networks, each three layers. First takes in 459 inputs and outputs 12, second and third take in 459 + (12*17) = 663 inputs.  Network 1: (459 * 150 + 150) + (150 * 150 + 150) + (150 * 150 + 150) + (150 * 12 + 12) = 116,112 Networks 2 and 3: (663 * 150 + 150) + (150 * 150 + 150) + (150 * 150 + 1",,,,Below a fruit fly,1.822e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://link.springer.com/protocol/10.1007/978-1-4939-6406-2_6,CC BY 4.0 (Epoch AI)
model,VD-LSTM+REAL Large,2016-11-04,"Salesforce Research,Stanford University",United States of America,Mixed or other,Language,parameters,51000000,51000000,Confident,51M (Table 3),,,,Below a fruit fly,2.13e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.01462,CC BY 4.0 (Epoch AI)
model,BIDAF,2016-11-05,"University of Washington,Allen Institute for AI",United States of America,Mixed or other,Language,parameters,2600000,2600000,Confident,"There are two similar models described in sections ""Models details"" citation from the paper about model for SQuAD ""The model has about 2.6 million parameters"" citation about model for cloze test ""The model architecture used for this task is very similar to that for SQuAD (Section 4) with only a few ",,,,Below a fruit fly,3.4686144e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.01603v6,CC BY 4.0 (Epoch AI)
model,NAS with base 8 and shared embeddings,2016-11-05,Google Brain,United States of America,Industry,Language,parameters,54000000,54000000,Confident,54M (Table 2),,,,Below a fruit fly,1.05e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.01578,CC BY 4.0 (Epoch AI)
model,NASv3 (CIFAR-10),2016-11-05,Google Brain,United States of America,Industry,Vision,parameters,37400000,37400000,Likely,Table 1,,,,Below a fruit fly,2.2e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.01578,CC BY 4.0 (Epoch AI)
model,DLDL (PASCAL),2016-11-06,University of Oxford,United Kingdom,Academia,Vision,parameters,564000000,564000000,Likely,Finetunes 4 versions of VGG VGG 16: 138000000 VGG 19: 144000000 Total 2*(138000000+144000000)=564000000 ,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Label-Distribution-Learning-With-Label-Gao-Xing/02567fd428a675ca91a0c6786f47f3e35881bcbd,CC BY 4.0 (Epoch AI)
model,DTN (Domain Transfer Network),2016-11-07,Facebook AI Research,France; United States of America,Industry,"Vision,Image generation",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.02200,CC BY 4.0 (Epoch AI)
model,DAC-CSR,2016-11-16,"Jiangnan University,University of Surrey",China; United Kingdom,Academia,Vision,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.05396,CC BY 4.0 (Epoch AI)
model,ResNeXt-101 (64×4d),2016-11-16,"University of California San Diego,Facebook",United States of America,Mixed or other,Vision,parameters,83000000,83000000,Confident,,,,,Fruit fly to honeybee,1.2e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.05431,CC BY 4.0 (Epoch AI)
model,ResNeXt-50,2016-11-16,"University of California San Diego,Facebook",United States of America,Mixed or other,Vision,parameters,25000000,25000000,Likely,"""If you’re thinking about ResNets, yes, they are related. ResNeXt-50 has 25M parameters (ResNet-50 has 25.5M).""  https://towardsdatascience.com/illustrated-10-cnn-architectures-95d78ace614d",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.05431,CC BY 4.0 (Epoch AI)
model,PolyNet,2016-11-17,Chinese University of Hong Kong (CUHK),Hong Kong,Academia,Vision,parameters,92000000,92000000,Likely,,,,,Fruit fly to honeybee,6.4e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.05725,CC BY 4.0 (Epoch AI)
model,Image-to-image cGAN,2016-11-21,University of California (UC) Berkeley,United States of America,Academia,"Vision,Image generation",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1611.07004,CC BY 4.0 (Epoch AI)
model,Elastic weight consolidation,2016-12-02,DeepMind,United Kingdom,Industry,"Vision,Games",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1612.00796,CC BY 4.0 (Epoch AI)
model,PointNet,2016-12-02,Stanford University,United States of America,Academia,3D modeling,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1612.00593,CC BY 4.0 (Epoch AI)
model,GAN-Advancer,2016-12-05,OpenAI,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.5555/3157096.3157346,CC BY 4.0 (Epoch AI)
model,HR-ResNet101,2016-12-13,Carnegie Mellon University (CMU),United States of America,Academia,Vision,parameters,44500000,44500000,Confident,ResNet 101 ,,,,Below a fruit fly,7.077e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Finding-Tiny-Faces-Hu-Ramanan/71f51e1b6691343ed031c1ed42efc96a7f1a0619,CC BY 4.0 (Epoch AI)
model,3DMM-CNN,2016-12-15,University of Southern California,United States of America,Academia,Vision,parameters,44500000,44500000,Confident,Based on ResNet 101 architecture,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1612.04904,CC BY 4.0 (Epoch AI)
model,EnhanceNet,2016-12-23,Max Planck Institute for Intelligent Systems,Germany,Academia,Vision,parameters,814464,814464,Confident,2*3*3*3*64+22*3*3*64*64=814464 24 CNN layers with 3x3 kernels and 64 channels and 3 input/output channels (see Table 1) ,,,,Below a fruit fly,1.3079231999999998e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/EnhanceNet%3A-Single-Image-Super-Resolution-Through-Sajjadi-Scholkopf/fddc32f3880688238847077fd927ab3025db7a6a,CC BY 4.0 (Epoch AI)
model,YOLOv2,2016-12-25,"University of Washington,Allen Institute for AI",United States of America,Mixed or other,Vision,parameters,51000000,51000000,,Source: https://resources.wolframcloud.com/NeuralNetRepository/resources/YOLO-V2-Trained-on-MS-COCO-Data_1,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1612.08242,CC BY 4.0 (Epoch AI)
model,DeepStack,2017-01-06,"University of Alberta,Charles University,Czech Technical University",Canada; Czechia,Academia,Games,parameters,2500000,2500000,Speculative,"Figure 3, p.9  source: https://docs.google.com/spreadsheets/d/1Kj4Q5WADcDXtUJLIOfGTCE3tGvxNczEMwyy8QtgSkHk/edit#gid=54587040&fvid=1361937389",,yes,,Below a fruit fly,1.446336e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1701.01724,CC BY 4.0 (Epoch AI)
model,OR-WideResNet,2017-01-07,"Duke University,University of Chinese Academy of Sciences",China; United States of America,Academia,Vision,parameters,18200000,18200000,Confident,18.2M for largest OR-WideResNet model.,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1701.01833v2,CC BY 4.0 (Epoch AI)
model,MoE-Multi,2017-01-23,"Jagiellonian University,Google Brain",Poland; United States of America,Mixed or other,Language,parameters,8700000000,8700000000,Confident,Table 5  https://arxiv.org/abs/1701.06538,,,,Honeybee to mouse,9.393905664e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1701.06538,CC BY 4.0 (Epoch AI)
model,DnCNN,2017-02-01,"Harbin Institute of Technology,Hong Kong Polytechnic University,ULSee Inc.,Xi’an Jiaotong University",China; Hong Kong,Mixed or other,"Vision,Image generation",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/abstract/document/7839189,CC BY 4.0 (Epoch AI)
model,Prototypical networks,2017-03-15,"University of Toronto,Twitter",Canada; United States of America,Mixed or other,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1703.05175,CC BY 4.0 (Epoch AI)
model,Mask R-CNN,2017-03-30,Facebook AI Research,France; United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1703.06870,CC BY 4.0 (Epoch AI)
model,WGAN-GP,2017-03-31,"Courant Institute of Mathematical Sciences,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms)",Canada; United States of America,Academia,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1704.00028,CC BY 4.0 (Epoch AI)
model,MobileNet,2017-04-17,Google,United States of America,Industry,Vision,parameters,4200000,4200000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1704.04861,CC BY 4.0 (Epoch AI)
model,DeepLab (2017),2017-04-27,"Johns Hopkins University,Google,University College London (UCL)",United Kingdom; United States of America,Mixed or other,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/abstract/document/7913730,CC BY 4.0 (Epoch AI)
model,Mnemonic Reader,2017-05-08,"Fudan University,Microsoft Research",China; United States of America,Mixed or other,Language,parameters,,,Confident,may be possible to estimate architecture description,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1705.02798v6,CC BY 4.0 (Epoch AI)
model,SRGAN,2017-05-25,Twitter,United States of America,Industry,"Vision,Image generation",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openaccess.thecvf.com/content_cvpr_2017/html/Ledig_Photo-Realistic_Single_Image_CVPR_2017_paper.html,CC BY 4.0 (Epoch AI)
model,Inflated 3D ConvNet,2017-06-01,"DeepMind,University of Oxford",United Kingdom,Mixed or other,3D modeling,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1705.07750,CC BY 4.0 (Epoch AI)
model,PointNet++,2017-06-07,Stanford University,United States of America,Academia,3D modeling,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1706.02413,CC BY 4.0 (Epoch AI)
model,Reading Twice for NLU,2017-06-08,DeepMind,United Kingdom,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1706.02596v3,CC BY 4.0 (Epoch AI)
model,EDSR,2017-06-10,Seoul National University,South Korea,Academia,"Vision,Image generation",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1707.02921,CC BY 4.0 (Epoch AI)
model,Transformer,2017-06-12,"Google Research,Google Brain",United States of America,Industry,Language,parameters,213000000,213000000,Confident,"This page suggests the transformer has 213M parameters.  ""Although there are others architectures that make use of attention layers, none achieves so good results so fast. Not only that, but the only model that can compite against Transformer is the Slicenet22, proposed just fifteen days before. It ",,,,Fruit fly to honeybee,7.4245248e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf,CC BY 4.0 (Epoch AI)
model,HRA,2017-06-13,"Maluuba,Microsoft",Canada; United States of America,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1706.04208,CC BY 4.0 (Epoch AI)
model,DeepLabV3,2017-06-17,Google,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1706.05587,CC BY 4.0 (Epoch AI)
model,NoisyNet-Dueling,2017-06-30,DeepMind,United Kingdom,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1706.10295v3,CC BY 4.0 (Epoch AI)
model,ShuffleNet v1,2017-07-03,Megvii Inc,China,Industry,Vision,parameters,2430000,2430000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1707.01083,CC BY 4.0 (Epoch AI)
model,DeepLoc,2017-07-07,"Technical University of Denmark,University of Copenhagen",Denmark,Academia,Biology,parameters,,,Likely,,,,,,5.781024e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://academic.oup.com/bioinformatics/article/33/21/3387/3931857,CC BY 4.0 (Epoch AI)
model,JFT,2017-07-10,"Google Research,Carnegie Mellon University (CMU)",United States of America,Mixed or other,Vision,parameters,44654504,44654504,Confident,"Uses ResNet-101 architecture, which has 44,654,504 parameters: https://resources.wolframcloud.com/NeuralNetRepository/resources/ResNet-101-Trained-on-ImageNet-Competition-Data/",,,,Below a fruit fly,8.43e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1707.02968,CC BY 4.0 (Epoch AI)
model,AWD-LSTM,2017-07-18,"DeepMind,University of Oxford",United Kingdom,Mixed or other,Language,parameters,24000000,24000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1707.05589,CC BY 4.0 (Epoch AI)
model,NASNet-A,2017-07-21,Google Brain,United States of America,Industry,Vision,parameters,89000000,89000000,,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1707.07012,CC BY 4.0 (Epoch AI)
model,PSPNet,2017-07-21,Chinese University of Hong Kong (CUHK),Hong Kong,Academia,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/8100143,CC BY 4.0 (Epoch AI)
model,ConvS2S (ensemble of 8 models),2017-07-25,Meta AI,United States of America,Industry,Language,parameters,,,Likely,,,,,,5.64e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1705.03122,CC BY 4.0 (Epoch AI)
model,GSM,2017-07-30,"Peking University,Microsoft Research",China; United States of America,Mixed or other,Language,parameters,,,Likely,It could be possible to estimate it from section 3.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aclanthology.org/P17-1018/,CC BY 4.0 (Epoch AI)
model,AWD-LSTM - 3-layer LSTM (tied) + continuous cache pointer (WT2),2017-08-07,Salesforce Research,United States of America,Industry,Language,parameters,33000000,33000000,Confident,33M (Table 2),,,,Below a fruit fly,2.97e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1708.02182,CC BY 4.0 (Epoch AI)
model,RetinaNet-R101,2017-08-07,Facebook AI Research,France; United States of America,Industry,Vision,parameters,53000000,53000000,Confident,source: table 2 in https://arxiv.org/pdf/1911.09070.pdf,,,,Below a fruit fly,2.065392e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1708.02002,CC BY 4.0 (Epoch AI)
model,RetinaNet-R50,2017-08-07,Facebook AI Research,France; United States of America,Industry,Vision,parameters,34000000,34000000,,source: table 2 in https://arxiv.org/pdf/1911.09070.pdf,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1708.02002,CC BY 4.0 (Epoch AI)
model,OpenAI TI7 DOTA 1v1,2017-08-11,OpenAI,United States of America,Industry,Games,parameters,,,Confident,"Section 4 states: ""we used a model with over 150 million parameters"" but this is for the 5v5 agent, not the 1v1.",,,,,6.046095222592002e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/research/dota-2,CC BY 4.0 (Epoch AI)
model,EI-REHN-1000D,2017-08-14,Korea Advanced Institute of Science and Technology (KAIST),South Korea,Academia,Language,parameters,19000000,19000000,Confident,19M (Table 4),,,,Below a fruit fly,1.06e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1708.04116,CC BY 4.0 (Epoch AI)
model,NeuMF (Pinterest),2017-08-16,"Shandong University,Texas A&M,National University of Singapore,Columbia University",China; Singapore; United States of America,Academia,Recommendation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1708.05031,CC BY 4.0 (Epoch AI)
model,Libratus,2017-08-19,Carnegie Mellon University (CMU),United States of America,Academia,Games,parameters,,,Likely,,,,,,5.51e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.ijcai.org/proceedings/2017/0772.pdf,CC BY 4.0 (Epoch AI)
model,GL-LWGC-AWD-MoS-LSTM + dynamic evaluation (WT2),2017-08-29,Ben-Gurion University of the Negev,Israel,Academia,Language,parameters,38000000,38000000,Confident,38M (Table 2),,,,Below a fruit fly,4.56e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1708.08863,CC BY 4.0 (Epoch AI)
model,SENet (ImageNet),2017-09-05,"Chinese Academy of Sciences,University of Oxford",China; United Kingdom,Academia,Vision,parameters,28100000,28100000,,Table 16,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1709.01507,CC BY 4.0 (Epoch AI)
model,PyramidNet,2017-09-06,Korea Advanced Institute of Science and Technology (KAIST),South Korea,Academia,Vision,parameters,26000000,26000000,Likely,best model had 26M params,,,,Below a fruit fly,2340000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1610.02915v4,CC BY 4.0 (Epoch AI)
model,ISS,2017-09-15,"Duke University,Microsoft",United States of America,Mixed or other,Language,parameters,11100000,11100000,Confident,11.1M (Table 2),,,,Below a fruit fly,3400000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1709.05027,CC BY 4.0 (Epoch AI)
model,LSTM + dynamic eval,2017-09-21,University of Edinburgh,United Kingdom,Academia,Language,parameters,50000000,50000000,Likely,table 2,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1709.07432,CC BY 4.0 (Epoch AI)
model,AWD-LSTM+WT+Cache+IOG (WT2),2017-09-26,NTT Communication Science Laboratories,Japan,Industry,Language,parameters,53000000,53000000,Confident,53M (Table 3),,,,Below a fruit fly,3180000000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1709.08907,CC BY 4.0 (Epoch AI)
model,Rainbow DQN,2017-10-06,DeepMind,United Kingdom,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1710.02298,CC BY 4.0 (Epoch AI)
model,AlphaGo Zero,2017-10-18,DeepMind,United Kingdom,Industry,Games,parameters,46400244,46400244,Confident,"Input size: 19*19*17=6137, internal dimension 19*19 1 conv block 17*3*3*256=39168 39 residual blocks 2*3*3*256*256=1179648 Policy head 256*2+2*19*19*(19*19+2)=262598 Value head 256*1+1*19*19*256+256*1=92928 Total: 39168+39*1179648+262598+92928=46400966",,,,Below a fruit fly,6.49439910290227e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/nature24270,CC BY 4.0 (Epoch AI)
model,AlphaGo Master,2017-10-19,DeepMind,United Kingdom,Industry,Games,parameters,,,Likely,,,,,,3.4100000001e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/nature24270,CC BY 4.0 (Epoch AI)
model,ProgressiveGAN,2017-10-27,Nvidia,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1710.10196,CC BY 4.0 (Epoch AI)
model,PhraseCond,2017-10-28,"Carnegie Mellon University (CMU),University of Pittsburgh",United States of America,Academia,Language,parameters,,,Confident,Unclear how many layers they use for self-attention (N) and fusion (L and K). Could calculate if these were known.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1710.10504v2,CC BY 4.0 (Epoch AI)
model,S-Norm,2017-10-29,"University of Washington,Allen Institute for AI",United States of America,Mixed or other,Language,parameters,,,Confident,Not stated. Probably obtainable from github: https://github.com/allenai/document-qa/tree/master,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1710.10723v2,CC BY 4.0 (Epoch AI)
model,DCN+,2017-10-31,Salesforce Research,United States of America,Industry,Language,parameters,,,Confident,Not directly repoted - It may be possible to extract number from: https://github.com/lmn-extracts/dcn_plus/tree/master/question_answering,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1711.00106v2,CC BY 4.0 (Epoch AI)
model,Fraternal dropout + AWD-LSTM 3-layer (WT2),2017-10-31,"Jagiellonian University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),University of Montreal / Université de Montréal",Canada; Poland,Academia,Language,parameters,34000000,34000000,Likely,34M (Table 2),,,,Below a fruit fly,3.06e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1711.00066,CC BY 4.0 (Epoch AI)
model,VQ-VAE,2017-11-02,DeepMind,United Kingdom,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1711.00937,CC BY 4.0 (Epoch AI)
model,"AWD-LSTM-MoS + dynamic evaluation (WT2, 2017)",2017-11-10,Carnegie Mellon University (CMU),United States of America,Academia,Language,parameters,35000000,35000000,Likely,35M (Table 2),,,,Below a fruit fly,3.36e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1711.03953,CC BY 4.0 (Epoch AI)
model,TriNet,2017-11-21,"Visual Computing Institute,RWTH Aachen University",Germany,Academia,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1703.07737,CC BY 4.0 (Epoch AI)
model,DL scaling Image,2017-12-01,Baidu,China,Industry,Vision,parameters,121000000,121000000,Confident,We test models with parameter counts ranging from 89K to 121M.,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Learning-Scaling-is-Predictable%2C-Empirically-Hestness-Narang/a1c922be467d1c0c64b963e65dae41778b81b2a0,CC BY 4.0 (Epoch AI)
model,DL scaling LM,2017-12-01,Baidu,China,Industry,Language,parameters,177000000,177000000,Speculative,,,yes,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Learning-Scaling-is-Predictable%2C-Empirically-Hestness-Narang/a1c922be467d1c0c64b963e65dae41778b81b2a0,CC BY 4.0 (Epoch AI)
model,DL scaling speech,2017-12-01,Baidu,China,Industry,Speech,parameters,193000000,193000000,Likely,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Deep-Learning-Scaling-is-Predictable%2C-Empirically-Hestness-Narang/a1c922be467d1c0c64b963e65dae41778b81b2a0,CC BY 4.0 (Epoch AI)
model,AlphaZero,2017-12-05,DeepMind,United Kingdom,Industry,Games,parameters,,,Likely,,,,,,1.0600000001e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1712.01815,CC BY 4.0 (Epoch AI)
model,T-DMCA,2018-01-30,Google Brain,United States of America,Industry,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1801.10198,CC BY 4.0 (Epoch AI)
model,ELMo,2018-02-01,"University of Washington,Allen Institute for AI",United States of America,Mixed or other,Language,parameters,94000000,94000000,Speculative,,,yes,,Fruit fly to honeybee,3300100000000010.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1802.05365,CC BY 4.0 (Epoch AI)
model,QRNN,2018-02-01,Salesforce Research,United States of America,Industry,Language,parameters,135000000,135000000,Likely,"Based on the details provided in the paper, the number of parameters in the QRNN model can be calculated as follows:  Embedding layer size: 400 Number of QRNN layers: 4 Number of nodes per QRNN layer: 2500 Vocabulary size: 267,735 (for WikiText-103 dataset) Adaptive softmax layer Parameters:  Embedd",,,,Fruit fly to honeybee,6.8866472e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mlsys.org/Conferences/doc/2018/50.pdf,CC BY 4.0 (Epoch AI)
model,IMPALA,2018-02-05,DeepMind,United Kingdom,Industry,Games,parameters,1600000,1600000,Confident,"""Figure 3 in the paper states that the large architecture has 1.6 million parameters. I am using the large model because it was the only one trained on all the Atari games at once, which seems like the most impressive task in the suite.""  Source: https://docs.google.com/spreadsheets/d/1Kj4Q5WADcDXtU",,,,Below a fruit fly,1.68e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1802.01561,CC BY 4.0 (Epoch AI)
model,DeepLabV3+,2018-02-07,Google,United States of America,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1802.02611v3,CC BY 4.0 (Epoch AI)
model,TCN (P-MNIST),2018-02-15,"Carnegie Mellon University (CMU),Intel Labs",United States of America,Mixed or other,Vision,parameters,42000,42000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openreview.net/forum?id=rk8wKk-R-,CC BY 4.0 (Epoch AI)
model,Chinese - English translation,2018-03-01,Microsoft,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.microsoft.com/en-us/research/publication/achieving-human-parity-on-automatic-chinese-to-english-news-translation/,CC BY 4.0 (Epoch AI)
model,4 layer QRNN (h=2500),2018-03-22,Salesforce Research,United States of America,Industry,Language,parameters,151000000,151000000,Likely,Table 6,,,,Fruit fly to honeybee,5.9158815e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1803.08240,CC BY 4.0 (Epoch AI)
model,YOLOv3,2018-04-08,University of Washington,United States of America,Academia,Vision,parameters,56933216,56933216,Likely,Feature extractor (ignoring biases) 32*3*3*3 + 64*3*3*32 + 32*1*1*64 + 64*3*3*32 + 128*3*3*64 + 2*(64*1*1*128 + 128*3*3*64) + 256*3*3*128 + 8*(128*1*1*256 + 256*3*3*128) + 512*3*3*256 +  8*(256*1*1*512 +  512*3*3*256) +  1024*3*3*512 +  4*(512*1*1*1024 + 1024*3*3*512) + 4*4*1024*1000  source: table ,,,,Fruit fly to honeybee,1.3416380824e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1804.02767,CC BY 4.0 (Epoch AI)
model,ResNeXt-101 32x48d,2018-05-02,Facebook,United States of America,Industry,Vision,parameters,829000000,829000000,Confident,Table 6 ,,,,Honeybee to mouse,8.74395e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1805.00932,CC BY 4.0 (Epoch AI)
model,Dropout-LSTM+Noise(Bernoulli) (WT2),2018-05-03,"Columbia University,New York University (NYU),Princeton University",United States of America,Academia,Language,parameters,51000000,51000000,Likely,"""The large network has 2 layers with 1500 hidden units each. This leads to a model complexity of 51 million parameters.""",,,,Below a fruit fly,1.27e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1805.01500,CC BY 4.0 (Epoch AI)
model,aLSTM(depth-2)+RecurrentPolicy (WT2),2018-05-22,"University of Manchester,Alan Turing Institute",United Kingdom,Mixed or other,Language,parameters,32000000,32000000,Confident,32M (Table 3),,,,Below a fruit fly,7.296e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1805.08574,CC BY 4.0 (Epoch AI)
model,GPT-1,2018-06-01,OpenAI,United States of America,Industry,Language,parameters,117000000,117000000,Likely,"""The model had 117M parameters in total.""  source: https://medium.com/walmartglobaltech/the-journey-of-open-ai-gpt-models-32d95b7b7fb2",,,,Fruit fly to honeybee,1.7578125e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/blog/language-unsupervised/,CC BY 4.0 (Epoch AI)
model,Relational Memory Core,2018-06-05,"DeepMind,University College London (UCL)",United Kingdom,Mixed or other,Language,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1806.01822,CC BY 4.0 (Epoch AI)
model,MobileNetV2,2018-06-18,Google,United States of America,Industry,Vision,parameters,3400000,3400000,,Rados,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ieeexplore.ieee.org/document/8578572,CC BY 4.0 (Epoch AI)
model,FTW (For The Win),2018-07-03,DeepMind,United Kingdom,Industry,Games,parameters,126001330,126001330,Speculative,"Architecture described in figure S11 of the supplement  The architecture includes modules for visual embedding, reward prediction, recurrent processing, policy, baseline and pixel control.  Input is 84x84x3 pixels as seen in figure S10 of the supplement  ""We elected to use a resolution of 84x84 pixe",,yes,,Fruit fly to honeybee,3.49e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1807.01281,CC BY 4.0 (Epoch AI)
model,Big-Little Net,2018-07-10,IBM,United States of America,Industry,Vision,parameters,77360000,77360000,Likely,Table 2,,,,Fruit fly to honeybee,2.46048e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1807.03848,CC BY 4.0 (Epoch AI)
model,Big-Little Net (speech),2018-07-10,IBM,United States of America,Industry,Speech,parameters,3320000,3320000,Speculative,table 3,,yes,,Below a fruit fly,4.290048e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1807.03848,CC BY 4.0 (Epoch AI)
model,AWD-LSTM-MoS+PDR + dynamic evaluation (WT2),2018-08-14,IBM,United States of America,Industry,Language,parameters,35000000,35000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1808.05908,CC BY 4.0 (Epoch AI)
model,Big Transformer for Back-Translation,2018-08-28,"Facebook AI Research,Google Brain",France; United States of America,Industry,Language,parameters,,,Likely,"""We re-implemented the Transformer model in py- torch using the fairseq toolkit.1 All experiments are based on the Big Transformer architecture with 6 blocks in the encoder and decoder. We use the same hyper-parameters for all experiments, i.e., word representations of size 1024, feed-forward layers",,,,,4.7808e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1808.09381,CC BY 4.0 (Epoch AI)
model,(ensemble): AWD-LSTM-DOC (fin) × 5 (WT2),2018-08-30,"NTT Communication Science Laboratories,Tohoku University",Japan,Mixed or other,Language,parameters,185000000,185000000,Confident,185M (table 8),,,,Fruit fly to honeybee,6.66e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1808.10143,CC BY 4.0 (Epoch AI)
model,NetSurfP-2.0,2018-09-10,"Technical University of Denmark,University of Copenhagen,Universidad Nacional de San Martín,AIMST University",Argentina; Denmark; Malaysia,Academia,Biology,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/311209v3.full,CC BY 4.0 (Epoch AI)
model,Transformer + Simple Recurrent Unit,2018-09-17,"ASAPP,Cornell University,Google,Princeton University",United States of America,Mixed or other,Language,parameters,90000000,90000000,Confident,"5-layer model, Table 3",,,,Fruit fly to honeybee,1.1e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1709.02755v5,CC BY 4.0 (Epoch AI)
model,"AWD-LSTM-MoS + dynamic evaluation (WT2, 2018)",2018-09-18,"Peking University,Microsoft Research Asia",China,Mixed or other,Language,parameters,35000000,35000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1809.06858,CC BY 4.0 (Epoch AI)
model,LSTM+NeuralCache,2018-09-24,"KU Leuven,ESAT - PSI,Apple",Belgium; United States of America,Mixed or other,Language,parameters,2100000,2100000,Likely,"Given:  Hidden size (H) = 512 Number of hidden layers (L) = 1 Input size (I) is not mentioned, so let's denote it as I The total number of parameters P in an LSTM can be calculated as follows:  P = 4 * ((I * H) + (H * H) + H)  This is for one layer of LSTM cells. Since the LSTM model described has o",,,,Below a fruit fly,982800000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1809.08826,CC BY 4.0 (Epoch AI)
model,Transformer (Adaptive Input Embeddings) WT103,2018-09-28,Facebook AI Research,France; United States of America,Industry,Language,parameters,247000000,247000000,Confident,Table 2,,,,Fruit fly to honeybee,4.47e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1809.10853,CC BY 4.0 (Epoch AI)
model,BERT-Large,2018-10-11,Google,United States of America,Industry,Language,parameters,340000000,340000000,Confident,340M,,,,Fruit fly to honeybee,2.85e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1810.04805,CC BY 4.0 (Epoch AI)
model,MetaMimic,2018-10-11,Google,United States of America,Industry,Robotics,parameters,22000000,22000000,,"""This representational demand motivates the introduction of high-capacity deep neural networks. We found the architecture, shown in Figure 3, with residual connections, 20 convolution layers with 512 channels for a total of 22 million parameters, and instance normalization to drastically improve per",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1810.05017,CC BY 4.0 (Epoch AI)
model,TrellisNet,2018-10-15,"Carnegie Mellon University (CMU),Bosch Center for Artificial Intelligence,Intel Labs",Germany; United States of America,Mixed or other,Language,parameters,180000000,180000000,Confident,"180M, Table 2",,,,Fruit fly to honeybee,2.78e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1810.06682,CC BY 4.0 (Epoch AI)
model,MemoReader,2018-10-31,"Samsung,Korea University",South Korea,Mixed or other,Language,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aclanthology.org/D18-1237/,CC BY 4.0 (Epoch AI)
model,Mesh-TensorFlow Transformer 2.9B (translation),2018-11-05,Google Brain,United States of America,Industry,Language,parameters,2900000000,2900000000,Likely,"2.9B from section 9.1 : ""On the WMT14 En-Fr translation tasks (3), we trained the models for 3 epochs. The largest model (2.9B parameters) was trained for 22 hours on a 128-core TPUv2 cluster.""",,,,Honeybee to mouse,6.84288e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1811.02084,CC BY 4.0 (Epoch AI)
model,Mesh-TensorFlow Transformer 4.9B (language),2018-11-05,Google Brain,United States of America,Industry,Language,parameters,4900000000,4900000000,Confident,4.9B from section 9.1 : ''The largest model (4.9B parameters) took 13 hours to train on a 512-core TPUv2 cluster.',,,,Honeybee to mouse,1.617408e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1811.02084,CC BY 4.0 (Epoch AI)
model,Fine-tuned-AWD-LSTM-DOC (fin),2018-11-12,Samsung R&D Institute Russia,Russia,Industry,Language,parameters,46000000,46000000,Confident,"This is the model trained on Penn Treebank, which uses as a base model the 23M model from Table 7 in https://aclanthology.org/D18-1489.pdf  They additionally train a discriminator with the same architecture, so total parameters is 2*23M = 46M",,,,Below a fruit fly,5.188e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1811.04623,CC BY 4.0 (Epoch AI)
model,Multi-cell LSTM,2018-11-15,University of Hyderabad,India,Academia,Language,parameters,7200000,7200000,Likely,"Based on the details in the paper, the number of parameters in the Multi-cell LSTM model can be calculated as follows:  The model has 2 hidden LSTM layers, each with 1500 hidden units  Each LSTM unit is a multi-cell LSTM with 10 memory cells per unit  For a standard LSTM layer with n hidden units:  ",,,,Below a fruit fly,2006640000000000.0,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1811.06477,CC BY 4.0 (Epoch AI)
model,GPipe (Transformer),2018-11-16,Google,United States of America,Industry,Language,parameters,6000000000,6000000000,Likely,Section 5: ,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1811.06965,CC BY 4.0 (Epoch AI)
model,SPN (ImageNet 128),2018-12-04,"Google Brain,DeepMind",United Kingdom; United States of America,Industry,Image generation,parameters,250000000,250000000,Confident,250M (Table 4),,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1812.01608,CC BY 4.0 (Epoch AI)
model,StyleGAN,2018-12-12,Nvidia,United States of America,Industry,Image generation,parameters,26200000,26200000,Confident,,,,,Below a fruit fly,3.93e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1812.04948,CC BY 4.0 (Epoch AI)
model,Transformer ELMo,2019-01-01,"Allen Institute for AI,University of Washington",United States of America,Mixed or other,Language,parameters,56000000,56000000,Confident,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.semanticscholar.org/paper/Dissecting-Contextual-Word-Embeddings%3A-Architecture-Peters-Neumann/ac11062f1f368d97f4c826c317bf50dcc13fdb59,CC BY 4.0 (Epoch AI)
model,Transformer-XL (257M),2019-01-09,"Carnegie Mellon University (CMU),Google Brain",United States of America,Mixed or other,Language,parameters,256999999,256999999,Speculative,"Transformer-XL Large, Table 1",,yes,,Fruit fly to honeybee,3.7832771e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1901.02860,CC BY 4.0 (Epoch AI)
model,MT-DNN,2019-01-31,Microsoft,United States of America,Industry,Language,parameters,330000000,330000000,Confident,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1901.11504,CC BY 4.0 (Epoch AI)
model,Hanabi 4 player,2019-02-01,"DeepMind,University of Oxford,Carnegie Mellon University (CMU),Google Brain",United Kingdom; United States of America,Mixed or other,Games,parameters,764000,764000,Confident,source: https://docs.google.com/spreadsheets/d/1Kj4Q5WADcDXtUJLIOfGTCE3tGvxNczEMwyy8QtgSkHk/edit#gid=54587040&fvid=1361937389,,,,Below a fruit fly,4.3e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1902.00506,CC BY 4.0 (Epoch AI)
model,SDE,2019-02-09,"Carnegie Mellon University (CMU),Google Brain,Monash University",Australia; United States of America,Mixed or other,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1902.03499,CC BY 4.0 (Epoch AI)
model,GPT-2 (1.5B),2019-02-14,OpenAI,United States of America,Industry,Language,parameters,1500000000,1500000000,Speculative,"""GPT-2 is a large transformer-based language model with 1.5 billion parameters""",,yes,,Honeybee to mouse,1.920000000001e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf,CC BY 4.0 (Epoch AI)
model,KataGo,2019-02-27,Jane Street,United States of America,Industry,Games,parameters,2500000,2500000,Speculative,https://arxiv.org/abs/2210.00849 gives parameter count for AlphaZero in Fig 1b.,,yes,,Below a fruit fly,2.32e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1902.10565,CC BY 4.0 (Epoch AI)
model,NMT Transformer 437M,2019-02-28,"Google,Bar-Ilan University",Israel; United States of America,Mixed or other,Language,parameters,437700000,437700000,Confident,"""Regarding the model, for these experiments we use a larger Transformer model with 6 layers in both the encoder and the decoder, model dimension set to 1024, hidden dimension size of 8192, and 16 attention heads. This results in a model with approximately 473.7M parameters.""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1903.00089,CC BY 4.0 (Epoch AI)
model,SciBERT,2019-03-26,Allen Institute for AI,United States of America,Mixed or other,Language,parameters,110000000,110000000,Confident,"110M size of bert base from https://huggingface.co/google-bert/bert-base-uncased relevant citation:  ""We use the original BERT code to train SCIBERT on our corpus with the same con- figuration and size as BERT-Base. We train 4 different versions of SCIBERT: (i) cased or un- cased and (ii) BASEVOCAB ",,,,Fruit fly to honeybee,8.926848e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1903.10676,CC BY 4.0 (Epoch AI)
model,Cross-lingual alignment,2019-04-04,"Tel Aviv University,Massachusetts Institute of Technology (MIT)",Israel; United States of America,Academia,Language,parameters,,,Speculative,,,,,,2.56e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1902.09492,CC BY 4.0 (Epoch AI)
model,True-Regularization+Finetune+Dynamic-Eval,2019-04-08,"Mobvoi,Williams College",China; United States of America,Mixed or other,Language,parameters,7000000,7000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1904.04163,CC BY 4.0 (Epoch AI)
model,WeNet (Penn Treebank),2019-04-08,Amazon,United States of America,Industry,Language,parameters,23000000,23000000,Likely,Table 1,,,,Below a fruit fly,7.30000001e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1904.03819,CC BY 4.0 (Epoch AI)
model,Transformer-XL + RMS dynamic eval,2019-04-17,University of Edinburgh,United Kingdom,Academia,Language,parameters,256999999,256999999,Confident,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1904.08378,CC BY 4.0 (Epoch AI)
model,BERT-Large-CAS (PTB+WT2+WT103),2019-04-20,Amazon,United States of America,Industry,Language,parameters,395000000,395000000,Likely,395M (Table 6),,,,Honeybee to mouse,1.5405e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1904.09408,CC BY 4.0 (Epoch AI)
model,MuseNet,2019-04-25,OpenAI,United States of America,Industry,Audio,parameters,2038431744,2038431744,Likely,From article: 72 Layers 24 Attention heads  Assumptions:  D_head = 64 (GPT-2) D_model = 24*64=1536 (based on D_head) D_mlp = 4*1536=6144  Attention: H*(W*(2*D+N)+N*M) 24*(1536*(2*64+64)+64*1536)=9437184 MLP: 1536*6144+6144*1536=18874368 Total parameters: 72*(9437184+18874368)=2038431744 ,,,,Honeybee to mouse,2.208301056e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/musenet/,CC BY 4.0 (Epoch AI)
model,Neuro-Symbolic Concept Learner,2019-04-26,"Massachusetts Institute of Technology (MIT),Tsinghua University,MIT-IBM Watson AI Lab,DeepMind",China; United Kingdom; United States of America,Mixed or other,"Vision,Language",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1904.12584,CC BY 4.0 (Epoch AI)
model,RaptorX-Contact,2019-05-02,Toyota Technological Institute at Chicago,United States of America,Academia,Biology,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/biorxiv/early/2019/05/02/624460.full.pdf,CC BY 4.0 (Epoch AI)
model,ResNeXt-101 Billion-scale,2019-05-02,Facebook AI,United States of America,Industry,Vision,parameters,193000000,193000000,,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1905.00546,CC BY 4.0 (Epoch AI)
model,AWD-LSTM-DRILL + dynamic evaluation† (WT2),2019-05-14,IDIAP,Switzerland,Academia,Language,parameters,34000000,34000000,Likely,"34M, Table 2",,,,Below a fruit fly,4.08e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1905.05513,CC BY 4.0 (Epoch AI)
model,CPC v2,2019-05-22,"DeepMind,University of California (UC) Berkeley",United Kingdom; United States of America,Mixed or other,Vision,parameters,303000000,303000000,,source: https://openai.com/blog/image-gpt/#rfref25d,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1905.09272,CC BY 4.0 (Epoch AI)
model,EfficientNet-L2,2019-05-28,Google,United States of America,Industry,Vision,parameters,480000000,480000000,,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1905.11946,CC BY 4.0 (Epoch AI)
model,DLRM-2020,2019-05-31,Facebook AI,United States of America,Industry,Recommendation,parameters,100000000000,100000000000,Confident,Figure 1  https://arxiv.org/abs/2104.05158,,,,Mouse to cat,4e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.00091,CC BY 4.0 (Epoch AI)
model,XLM,2019-06-01,Facebook,United States of America,Industry,Language,parameters,665000000,665000000,,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1901.07291,CC BY 4.0 (Epoch AI)
model,XLNet,2019-06-01,"Carnegie Mellon University (CMU),Google Brain",United States of America,Mixed or other,Language,parameters,340000000,340000000,Confident,"Same size as BERT-Large, which was 340M",,,,Fruit fly to honeybee,6.19e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.08237,CC BY 4.0 (Epoch AI)
model,Transformer-XL Large + Phrase Induction,2019-06-04,"Massachusetts Institute of Technology (MIT),University of Illinois Urbana-Champaign (UIUC)",United States of America,Academia,Language,parameters,256999999,256999999,Speculative,,,yes,,Fruit fly to honeybee,3.7848651e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.01702,CC BY 4.0 (Epoch AI)
model,AWD-LSTM + MoS + Partial Shuffled,2019-06-10,University of Texas at Austin,United States of America,Academia,Language,parameters,35000000,35000000,Confident,35M (Table 2),,,,Below a fruit fly,3.15e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.03805,CC BY 4.0 (Epoch AI)
model,Char-CNN-BiLSTM,2019-06-13,Capital One,United States of America,Industry,Language,parameters,,,Confident,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.05678,CC BY 4.0 (Epoch AI)
model,FixRes ResNeXt-101 WSL,2019-06-14,Facebook AI,United States of America,Industry,Vision,parameters,829000000,829000000,,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.06423,CC BY 4.0 (Epoch AI)
model,PG-SWGAN,2019-06-15,ETH Zurich,Switzerland,Academia,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openaccess.thecvf.com/content_CVPR_2019/html/Wu_Sliced_Wasserstein_Generative_Models_CVPR_2019_paper.html,CC BY 4.0 (Epoch AI)
model,LaNet-L (CIFAR-10),2019-06-17,"Brown University,Facebook",United States of America,Mixed or other,Vision,parameters,44100000,44100000,Confident,44.1M,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1906.06832,CC BY 4.0 (Epoch AI)
model,Walking Minotaur robot,2019-06-19,"University of California (UC) Berkeley,Google Brain",United States of America,Mixed or other,Robotics,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1812.11103,CC BY 4.0 (Epoch AI)
model,RoBERTa Large,2019-07-01,"Facebook,University of Washington",United States of America,Mixed or other,Language,parameters,355000000,355000000,Confident,355M  https://github.com/facebookresearch/fairseq/blob/main/examples/roberta/README.md,,,,Fruit fly to honeybee,8.5067e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1907.11692,CC BY 4.0 (Epoch AI)
model,BigBiGAN,2019-07-04,Google,United States of America,Industry,"Vision,Image generation",parameters,86000000,86000000,,https://openai.com/blog/image-gpt/#rfref53,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1907.02544,CC BY 4.0 (Epoch AI)
model,Pluribus,2019-07-11,Facebook AI Research,France; United States of America,Industry,Games,parameters,,,Likely,,,,,,6.6e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.science.org/cms/asset/910714a7-ee2a-486e-9970-42fb893b08d9/pap.pdf,CC BY 4.0 (Epoch AI)
model,EN^2AS with performance reward,2019-07-22,"Beijing Institute of Technology,University of Technology Sydney,Monash University",Australia; China,Academia,Language,parameters,23000000,23000000,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1907.09109,CC BY 4.0 (Epoch AI)
model,trRosetta,2019-08-22,"Nankai University,University of Washington,Tianjin University,Harvard University",China; United States of America,Academia,Biology,parameters,,,Confident,,,,,,3.8047968e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.pnas.org/doi/10.1073/pnas.1914677117,CC BY 4.0 (Epoch AI)
model,"Mogrifier (d2, MoS2, MC) + dynamic eval",2019-09-04,"DeepMind,University of Oxford",United Kingdom,Mixed or other,Language,parameters,35000000,35000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.01792,CC BY 4.0 (Epoch AI)
model,UDSMProt,2019-09-04,Fraunhofer Heinrich Hertz Institute,Germany,Mixed or other,Biology,parameters,28303800,28303800,Likely,"Python code:   # Given LSTM parameters emb_sz = 400  # embedding size, typically equal to the input size for the first layer nh = 1150     # number of hidden units nl = 3        # number of layers  # The formula for a single LSTM layer parameters is: # P = 4 * ((input_dim + hidden_dim) * hidden_dim ",,,,Below a fruit fly,6.37e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/704874v2.full.pdf,CC BY 4.0 (Epoch AI)
model,Megatron-BERT,2019-09-17,Nvidia,United States of America,Industry,Language,parameters,3900000000,3900000000,Confident,Table 4,,,,Honeybee to mouse,2.2e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.08053,CC BY 4.0 (Epoch AI)
model,Megatron-LM (1.2B),2019-09-17,Nvidia,United States of America,Industry,Language,parameters,1200000000,1200000000,Confident,Table 1 in https://arxiv.org/pdf/1909.08053,,,,Honeybee to mouse,1.1304e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.08053,CC BY 4.0 (Epoch AI)
model,Megatron-LM (8.3B),2019-09-17,Nvidia,United States of America,Industry,Language,parameters,8300000000,8300000000,Likely,Source: https://lair.lighton.ai/akronomicon/  Archived source: https://web.archive.org/web/20211220142906/https://lair.lighton.ai/akronomicon/  Data also available on GitHub: https://github.com/lightonai/akronomicon/blob/main/akrodb/NVIDIA/Megatron-LM.json,,,,Honeybee to mouse,9.1e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.08053,CC BY 4.0 (Epoch AI)
model,Adaptive Inputs + LayerDrop,2019-09-25,"Facebook AI Research,LORIA",France; United States of America,Mixed or other,Language,parameters,423000000,423000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.11556,CC BY 4.0 (Epoch AI)
model,ALBERT,2019-09-26,"Toyota Technological Institute at Chicago,Google Research",United States of America,Mixed or other,Language,parameters,18000000,18000000,Confident,Section 3.2 of paper,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.11942,CC BY 4.0 (Epoch AI)
model,AlphaX-1,2019-10-02,"Facebook AI Research,Brown University",France; United States of America,Mixed or other,Vision,parameters,5400000,5400000,Confident,"Table 3: multiadds for AlphaX-1 579M, parameters 5.4M",,,,Below a fruit fly,8.89344e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1903.11059,CC BY 4.0 (Epoch AI)
model,DistilBERT,2019-10-02,Hugging Face,United States of America,Industry,Language,parameters,66000000,66000000,Confident,Table 3,,,,Fruit fly to honeybee,1.24416e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1910.01108,CC BY 4.0 (Epoch AI)
model,M4-50B,2019-10-11,Google,United States of America,Industry,Language,parameters,50000000000,50000000000,Confident,"(sparse architecture)  ""By modifying the Transformer architecture through the substitution of the vanilla feed-forward layers with sparsely-gated mixture of experts, we drastically scale up the model capacity, allowing us to successfully train and pass 50 billion parameters, which further improved t",yes,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.research.google/2019/10/exploring-massively-multilingual.html,CC BY 4.0 (Epoch AI)
model,T5-11B,2019-10-23,Google,United States of America,Industry,Language,parameters,11000000000,11000000000,Confident,The full 11-billion parameter model,,,,Honeybee to mouse,3.3e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1910.10683,CC BY 4.0 (Epoch AI)
model,T5-3B,2019-10-23,Google,United States of America,Industry,Language,parameters,2800000000,2800000000,Confident,"page 37, 3B and 11B. ""To further explore what kind of performance is possible when using larger models, we consider two additional variants. In both cases, we use d_model = 1024, a 24 layer encoder and decoder, and dkv = 128. For the “3B” variant, we use dff = 16,384 with 32-headed attention, which ",,,,Honeybee to mouse,9.0000000001e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1910.10683,CC BY 4.0 (Epoch AI)
model,BART-large,2019-10-29,Facebook AI,United States of America,Industry,Language,parameters,406291456,406291456,Likely,"""In total, BART contains roughly 10% more parameters than the equivalently sized BERT model.""  I counted the parameters in the huggingface model https://huggingface.co/facebook/bart-large/tree/main  from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained(""facebook",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1910.13461,CC BY 4.0 (Epoch AI)
model,AlphaStar,2019-10-30,DeepMind,United Kingdom,Industry,Games,parameters,139000000,139000000,Confident,"AlphaStar has 139 million weights, but only 55 million weights are required during inference.",,,,Fruit fly to honeybee,1.0773400001e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.deepmind.com/blog/alphastar-grandmaster-level-in-starcraft-ii-using-multi-agent-reinforcement-learning,CC BY 4.0 (Epoch AI)
model,Base LM + kNN LM + Continuous Cache,2019-11-01,"Stanford University,Facebook AI Research",France; United States of America,Mixed or other,Language,parameters,247000000,247000000,Likely,"""we take the exact architecture and optimization described by Baevski & Auli (2019) and use it to create a kNN-LM for inference. This model consists of 16 layers, each with 16 self-attention heads, 1024 dimensional hidden states, and 4096 dimensional feedforward layers, amounting to 247M trainable p",,,,Fruit fly to honeybee,3.05292e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.00172,CC BY 4.0 (Epoch AI)
model,XLM-RoBERTa,2019-11-05,Facebook AI,United States of America,Industry,Language,parameters,550000000,550000000,Confident,"The number of parameters in the model is specified as ""550M params"" for XLM-R.",,,,Honeybee to mouse,2.076e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.02116,CC BY 4.0 (Epoch AI)
model,CamemBERT,2019-11-10,"Facebook,INRIA,Sorbonne University",France; United States of America,Mixed or other,Language,parameters,335000000,335000000,Confident,"CamemBERT Large, Table 4",,,,Fruit fly to honeybee,8.3e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.03894,CC BY 4.0 (Epoch AI)
model,Sandwich Transformer,2019-11-10,"Allen Institute for AI,Facebook AI Research",France; United States of America,Mixed or other,Language,parameters,209000000,209000000,Speculative,"209M ""All of our experiments use the same hyperparameters as Baevski and Auli’s original model.""",,yes,,Fruit fly to honeybee,2.3504093e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.03864,CC BY 4.0 (Epoch AI)
model,Noisy Student (L2),2019-11-11,"Carnegie Mellon University (CMU),Google",United States of America,Mixed or other,Vision,parameters,480000000,480000000,Confident,,,,,Honeybee to mouse,2.612e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.04252v4,CC BY 4.0 (Epoch AI)
model,MoCo,2019-11-13,Facebook AI,United States of America,Industry,"Vision,Image generation",parameters,375000000,375000000,,https://openai.com/blog/image-gpt/#rfref53,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.05722,CC BY 4.0 (Epoch AI)
model,MuZero,2019-11-19,DeepMind,United Kingdom,Industry,Games,parameters,36864000,36864000,Speculative,"Both the representation and dynamics function use the same architecture asAlphaZero, but with 16 instead of20 residual blocks [15]. We use 3x3 kernels and 256 hidden planes for each convolution.  Previous downsampling: •  1 convolution with stride 2 and 128 output planes, output resolution 48x48.•  ",,yes,,Below a fruit fly,4.8e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.08265v2,CC BY 4.0 (Epoch AI)
model,Transformer - LibriVox + Decoding/Rescoring,2019-11-19,Facebook,United States of America,Industry,Speech,parameters,296000000,296000000,Confident,Table 2,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.08460v3,CC BY 4.0 (Epoch AI)
model,Photo-Geometric Autoencoder,2019-11-25,University of Oxford,United Kingdom,Academia,"3D modeling,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.11130,CC BY 4.0 (Epoch AI)
model,Transformer-XL DeFINE (141M),2019-11-27,"University of Washington,Allen Institute for AI",United States of America,Mixed or other,Language,parameters,141000000,141000000,Speculative,Table 2b,,yes,,Fruit fly to honeybee,1.74276e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.12385,CC BY 4.0 (Epoch AI)
model,StyleGAN2,2019-12-03,"Nvidia,Aalto University",Finland; United States of America,Mixed or other,Image generation,parameters,30000000,30000000,Confident,"30M for the generator, 29M for the discriminator",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/1912.04958,CC BY 4.0 (Epoch AI)
model,StarGAN v2,2019-12-04,"NAVER,Yonsei University,Swiss Federal Institute of Technology",South Korea; Switzerland,Mixed or other,"Vision,Image generation",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1912.01865,CC BY 4.0 (Epoch AI)
model,MMLSTM (PTB),2019-12-05,"Beijing University of Posts and Telecommunications,University of West London",China; United Kingdom,Academia,Language,parameters,21300000,21300000,Likely,Table II,,,,Below a fruit fly,5.8298782e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://repository.uwl.ac.uk/id/eprint/6490/1/Loo_etal_IEEE_TNNLS_2019_Major-minor_long_short-term_memory_for_word-level_language_model.pdf,CC BY 4.0 (Epoch AI)
model,MMLSTM (WT-2),2019-12-05,"Beijing University of Posts and Telecommunications,University of West London",China; United Kingdom,Academia,Language,parameters,32300000,32300000,Likely,Table III,,,,Below a fruit fly,1.938e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://repository.uwl.ac.uk/id/eprint/6490/1/Loo_etal_IEEE_TNNLS_2019_Major-minor_long_short-term_memory_for_word-level_language_model.pdf,CC BY 4.0 (Epoch AI)
model,OpenAI Five,2019-12-13,OpenAI,United States of America,Industry,Games,parameters,159000000,159000000,Confident,"""We define a policy (π) as a function from the history of observations to a probability distribution over actions, which we parameterize as a recurrent neural network with approximately 159 million parameters (θ)."" pg. 3 of paper  source: https://docs.google.com/spreadsheets/d/1Kj4Q5WADcDXtUJLIOfGTC",,,,Fruit fly to honeybee,6.7e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1912.06680,CC BY 4.0 (Epoch AI)
model,OpenAI Five Rerun,2019-12-13,OpenAI,United States of America,Industry,Games,parameters,159000000,159000000,Confident,"""We define a policy (π) as a function from the history of observations to a probability distribution over actions, which we parameterize as a recurrent neural network with approximately 159 million parameters (θ)."" pg. 3 of paper  source: https://docs.google.com/spreadsheets/d/1Kj4Q5WADcDXtUJLIOfGTC",,,,Fruit fly to honeybee,1.3e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/dota-2.pdf,CC BY 4.0 (Epoch AI)
model,DD-PPO,2019-12-19,"Georgia Institute of Technology,Facebook AI Research,Oregon State University,Simon Fraser University",Canada; France; United States of America,Mixed or other,Robotics,parameters,,,Likely,"no parameter count but some architecture details: ""The policy is parameterized by a 2-layer LSTM with a 512-dimensional hidden state. It takes three inputs: the previous action, the target relative to the current state, and the output of the visual encoder. The LSTM’s output is used to produce a sof",,,,,7.8e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://openreview.net/forum?id=H1gX8C4YPr
https://arxiv.org/abs/1911.00357",CC BY 4.0 (Epoch AI)
model,Big Transfer (BiT-L),2019-12-24,Google Brain,United States of America,Industry,Vision,parameters,928000000,928000000,,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1912.11370,CC BY 4.0 (Epoch AI)
model,AlphaFold,2020-01-15,DeepMind,United Kingdom,Industry,Biology,parameters,16340840,16340840,Speculative,"""Neural network hyperparameters"" section of https://www.nature.com/articles/s41586-019-1923-7: “7 × 4 Blocks with 256 channels, cycling through dilations 1, 2, 4, 8” “48 × 4 Blocks with 128 channels, cycling through dilations 1, 2, 4, 8”  ""Distogram prediction"" section: ""For the final layer, a posit",,yes,,Below a fruit fly,1e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-019-1923-7,CC BY 4.0 (Epoch AI)
model,ContextNet + Noisy Student,2020-01-19,Google,United States of America,Industry,Speech,parameters,,,Confident,,,,,,8.16e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.09629v2,CC BY 4.0 (Epoch AI)
model,Meena,2020-01-28,Google Brain,United States of America,Industry,Language,parameters,2600000000,2600000000,Confident,"""We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token.""",,,,Honeybee to mouse,1.12e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2001.09977,CC BY 4.0 (Epoch AI)
model,Theseus 6/768,2020-02-07,"University of California San Diego,Beihang University,Microsoft",China; United States of America,Mixed or other,Language,parameters,66000000,66000000,Confident,"66M, Table 1",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2002.02925,CC BY 4.0 (Epoch AI)
model,Perceiver IO (optical flow),2020-02-08,DeepMind,United Kingdom,Industry,"Multimodal,Language,Vision",parameters,27900000,27900000,Confident,"Optical flow model (SOTA) was 27.9M params. There are other, larger models described in this paper, e.g. for language.  ""For the pixel- and patch-based models, total computational complexity for a forward pass on a 368 × 496 image is roughly 987 billion FLOPs, and there are roughly 27.9 million para",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2107.14795,CC BY 4.0 (Epoch AI)
model,TaLK Convolution,2020-02-08,Carleton University,Canada,Academia,Language,parameters,240000000,240000000,Likely,"Table 5  ""For the language model, we followed the same configuration as Baevski & Auli (2019). We used 17 decoding layers, each layer with a 1024 hidden size, a 4096 feed-forward hidden size and 8 heads. The adaptive input factor was set to 4.""",,,,Fruit fly to honeybee,2.6990346e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2002.03184,CC BY 4.0 (Epoch AI)
model,ALBERT-xxlarge,2020-02-09,"Toyota Technological Institute at Chicago,Google",United States of America,Mixed or other,Language,parameters,235000000,235000000,Speculative,,,yes,,Fruit fly to honeybee,2.39e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1909.11942,CC BY 4.0 (Epoch AI)
model,SimCLR,2020-02-13,Google Brain,United States of America,Industry,Vision,parameters,375000000,375000000,,source: https://openai.com/blog/image-gpt/,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2002.05709,CC BY 4.0 (Epoch AI)
model,Turing-NLG,2020-02-13,Microsoft,United States of America,Industry,Language,parameters,17000000000,17000000000,Likely,,,,,Honeybee to mouse,1.57e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/,CC BY 4.0 (Epoch AI)
model,FFN SwiGLU,2020-02-14,Google,United States of America,Industry,Language,parameters,220000000,220000000,Confident,"""We use the same code base, model architecture, and training task as the base model from [Raffel et al.,2019]. The encoder and decoder each consist of 12 layers, with dmodel = 768. For the attention layers, h = 12 and dk = dv = 64. The FFN layers have hidden size df f = 3072.""  [Raffel et al.,2019]:",,,,Fruit fly to honeybee,3.3687317e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2002.05202,CC BY 4.0 (Epoch AI)
model,Feedback Transformer,2020-02-21,"LORIA,University of Lorraine,Facebook AI Research",France; United States of America,Mixed or other,Language,parameters,126000000,126000000,Speculative,Table 3 shows 126M.  There is another instance of the Feedback Transformer mentioned in Table 9 with 139M parameters.,,yes,,Fruit fly to honeybee,7.690547e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2002.09402,CC BY 4.0 (Epoch AI)
model,TCAN (WT2),2020-02-28,"Nanjing University,Ant Group",China,Mixed or other,Language,parameters,33000000,33000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2002.12530,CC BY 4.0 (Epoch AI)
model,TransformerXL + spectrum control,2020-03-11,"University of California Los Angeles (UCLA),JD.com",China; United States of America,Mixed or other,Language,parameters,151000000,151000000,Speculative,"151M (Table 2)  "" On the large WikiText-103 dataset, we implement our method based on the state-of-the-art Transformer-XL based models (Dai et al., 2019). We follow the same settings reported in (Dai et al., 2019), and our implementation is based on the official code for Transformer-XL.""",,yes,,Fruit fly to honeybee,2.6289761e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openreview.net/forum?id=ByxY8CNtvr,CC BY 4.0 (Epoch AI)
model,Routing Transformer (WT-103),2020-03-12,Google Research,United States of America,Industry,Language,parameters,79500000,79500000,Confident,,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2003.05997,CC BY 4.0 (Epoch AI)
model,Tensor-Transformer(1core)+PN (WT103),2020-03-17,University of California (UC) Berkeley,United States of America,Academia,Language,parameters,85300000,85300000,Confident,"six layers tensorized transformer core-1 for Wikitext-103, following (Ma et al., 2019).  https://arxiv.org/abs/1906.09777",,,,Fruit fly to honeybee,1.58e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2003.07845,CC BY 4.0 (Epoch AI)
model,ELECTRA,2020-03-23,"Stanford University,Google,Google Brain",United States of America,Mixed or other,Language,parameters,335000000,335000000,Likely,https://github.com/google-research/electra,,,,Fruit fly to honeybee,3.1e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2003.10555v1,CC BY 4.0 (Epoch AI)
model,MetNet,2020-03-24,Google,United States of America,Industry,Earth science,parameters,225000000,225000000,Speculative,""" In total this setting for MetNet has 225M parameters.""",,yes,,Fruit fly to honeybee,9.510912e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2003.12140,CC BY 4.0 (Epoch AI)
model,Agent57,2020-03-30,DeepMind,United Kingdom,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2003.13350,CC BY 4.0 (Epoch AI)
model,CURL,2020-04-08,University of California (UC) Berkeley,United States of America,Academia,Games,parameters,907264,907264,,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2004.04136v4,CC BY 4.0 (Epoch AI)
model,Go-explore,2020-04-27,"Uber AI,OpenAI",United States of America,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2004.12919,CC BY 4.0 (Epoch AI)
model,Once for All,2020-04-29,"MIT-IBM Watson AI Lab,Massachusetts Institute of Technology (MIT),IBM",United States of America,Mixed or other,Vision,parameters,7700000,7700000,Confident,"""Since all of these sub-networks share the same weights (i.e., Wo) (Cheung et al., 2019), we only require 7.7M parameters to store all of them. Without sharing, the total model size will be prohibitive""",,,,Below a fruit fly,6.237e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1908.09791,CC BY 4.0 (Epoch AI)
model,UnifiedQA,2020-05-02,"Allen Institute for AI,University of Washington",United States of America,Mixed or other,Language,parameters,11000000000,11000000000,Confident,"11B from appendix A.2 : Model sizes: ""Most of the experiments are done on T5(11B) which has 11 billion parameters. We also report experiments with BART (large) with 440 million parameters.""",,,,Honeybee to mouse,1.65e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.00700v3,CC BY 4.0 (Epoch AI)
model,NAS+ESS (23M),2020-05-06,"Northeastern University (China),NiuTrans Research,Kingsoft",China,Mixed or other,Language,parameters,23000000,23000000,Confident,,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/2005.02593,CC BY 4.0 (Epoch AI)
model,ContextNet,2020-05-07,Google,United States of America,Industry,Speech,parameters,112700000,112700000,Likely,Table 5,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.03191v3,CC BY 4.0 (Epoch AI)
model,Conformer,2020-05-16,Google,United States of America,Industry,Speech,parameters,118800000,118800000,Confident,118.8M for Conformer(L),,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.08100v1,CC BY 4.0 (Epoch AI)
model,Retrieval-Augmented Generator,2020-05-22,"Facebook,New York University (NYU),University College London (UCL)",United Kingdom; United States of America,Mixed or other,Language,parameters,626000000,626000000,Confident,"""Our RAG models contain the trainable parameters for the BERT-base query and document encoder of DPR, with 110M parameters each (although we do not train the document encoder ourselves) and 406M trainable parameters from BART-large, 406M parameters, making a total of 626M trainable parameters""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.11401v4,CC BY 4.0 (Epoch AI)
model,DETR,2020-05-26,Facebook,United States of America,Industry,Vision,parameters,60000000,60000000,Confident,60M per Table 1,,,,Fruit fly to honeybee,4e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.12872,CC BY 4.0 (Epoch AI)
model,GPT-3 175B (davinci),2020-05-28,OpenAI,United States of America,Industry,Language,parameters,174600000000,174600000000,Confident,"""we train GPT-3, an autoregressive language model with 175 billion parameters"" Rather, it's 174.6 billion. Table D.1: 174,600 million parameters",,,,Mouse to cat,3.14e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2005.14165,CC BY 4.0 (Epoch AI)
model,GShard (dense),2020-06-30,Google,United States of America,Industry,Language,parameters,2300000000,2300000000,Confident,"""Our best quality dense single Transformer model (2.3B parameters) achieving ∆BLEU of 6.1, was trained with GPipe [15] on 2048 TPU v3 cores for 6 weeks or total of 235.5 TPU v3 core-years.""",,,,Honeybee to mouse,4.765e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2006.16668,CC BY 4.0 (Epoch AI)
model,SemExp,2020-07-02,"Carnegie Mellon University (CMU),Facebook AI Research",France; United States of America,Mixed or other,Robotics,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://proceedings.neurips.cc/paper/2020/file/2c75cf2681788adaca63aa95ae028b22-Paper.pdf
https://arxiv.org/abs/2007.00643",CC BY 4.0 (Epoch AI)
model,Hopfield Networks (2020),2020-07-16,"Johannes Kepler University Linz,Institute of Advanced Research in Artificial Intelligence,University of Oslo",Austria; Norway,Academia,"Biology,Vision,Language,Medicine",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2008.02217,CC BY 4.0 (Epoch AI)
model,EfficientDet,2020-07-27,Google Brain,United States of America,Industry,Vision,parameters,77000000,77000000,,"""EfficientDet-D7 achieves stateof-the-art 55.1 AP on COCO test-dev with 77M parameters and 410B FLOPs""",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.09070,CC BY 4.0 (Epoch AI)
model,DeLighT,2020-08-03,"University of Washington,Allen Institute for AI,Facebook AI Research",France; United States of America,Mixed or other,Language,parameters,99000000,99000000,Likely,99M (Table 4b),,,,Fruit fly to honeybee,3.8016e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2008.00623,CC BY 4.0 (Epoch AI)
model,ERNIE-GEN (large),2020-08-06,Baidu,China,Industry,Language,parameters,340000000,340000000,Speculative,"""We train a base model ERNIEGENBASE (L=12, H=768, A=12, Total Parameters=110M)1 and a large model ERNIE-GENLARGE (L=24, H=1024, A=16, Total Parameters=340M) with parameters initialized by BERTBASE and BERTLARGE respectively""",,yes,,Fruit fly to honeybee,2e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2001.11314,CC BY 4.0 (Epoch AI)
model,ProBERTa,2020-09-01,"University of Illinois Urbana-Champaign (UIUC),Reed College",United States of America,Academia,Biology,parameters,44000000,44000000,Confident,"""In total, our model has approximately 44M trainable parameters.""",,,,Below a fruit fly,9.72e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://dl.acm.org/doi/10.1145/3388440.3412467,CC BY 4.0 (Epoch AI)
model,LUKE,2020-10-02,"University of Washington,National Institute of Informatics",Japan; United States of America,Academia,Language,parameters,483000000,483000000,Likely,"""The total number of parameters is approximately 483 M, consisting of 355 M in RoBERTa and 128 M in our entity embeddings""",,,,Honeybee to mouse,1.8144e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2010.01057v1,CC BY 4.0 (Epoch AI)
model,Conformer + Wav2vec 2.0 + Noisy Student,2020-10-20,"Google,Google Research,Google Brain",United States of America,Industry,Speech,parameters,1000000000,1000000000,Confident,1B for XXL model,,,,Honeybee to mouse,7.6e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2010.10504v2,CC BY 4.0 (Epoch AI)
model,mT5-XXL,2020-10-20,"Google,Google Research",United States of America,Industry,Language,parameters,13000000000,13000000000,Confident,13 billion,,,,Honeybee to mouse,8.2e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aclanthology.org/2021.naacl-main.41/,CC BY 4.0 (Epoch AI)
model,German ELECTRA Large,2020-10-21,"deepset,Bayerische Staatsbibliothek Muenchen",Germany,Mixed or other,Language,parameters,335000000,335000000,Confident,335M from Table 5,,,,Fruit fly to honeybee,1.42829568e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2010.10906,CC BY 4.0 (Epoch AI)
model,ViT-Base/32,2020-10-22,Google Brain,United States of America,Industry,Vision,parameters,86000000,86000000,,Table 1 https://arxiv.org/pdf/2010.11929.pdf,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2010.11929,CC BY 4.0 (Epoch AI)
model,ViT-Huge/14,2020-10-22,"Google Brain,Google Research",United States of America,Industry,Vision,parameters,632000000,632000000,Confident,Table 1 https://arxiv.org/pdf/2010.11929.pdf,,,,Honeybee to mouse,4.262e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2010.11929,CC BY 4.0 (Epoch AI)
model,wave2vec 2.0 LARGE,2020-10-22,Facebook,United States of America,Industry,Speech,parameters,317000000,317000000,Confident,"Section 5.1: ""We consider two model sizes: BASE (95m parameters) and LARGE (317m parameters) ",,,,Fruit fly to honeybee,3.87072e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2006.11477,CC BY 4.0 (Epoch AI)
model,KEPLER,2020-11-23,"Tsinghua University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),HEC,CIFAR AI Research,Princeton University,University of Montreal / Université de Montréal",Canada; China; France; United States of America,Mixed or other,Language,parameters,125000000,125000000,Confident,,,,,Fruit fly to honeybee,1.66e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/1911.06136,CC BY 4.0 (Epoch AI)
model,AlphaFold 2,2020-11-30,DeepMind,United Kingdom,Industry,Biology,parameters,93000000,93000000,Likely,"https://arxiv.org/abs/2207.05477 reimplements AlphaFold 2 in a more efficient way, and states there are 93M parameters in the original version (Table 1)",,,,Fruit fly to honeybee,2.99e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-021-03819-2,CC BY 4.0 (Epoch AI)
model,CPM-Large,2020-12-01,"Tsinghua University,Beijing Academy of Artificial Intelligence / BAAI",China,Academia,Language,parameters,2600000000,2600000000,Confident,"""To the best of our knowledge, CPM, with 2.6 billion parameters and 100GB Chinese training data, is the largest Chinese pre-trained language mode""",,,,Honeybee to mouse,2.6052e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2012.00413,CC BY 4.0 (Epoch AI)
model,ESM1b,2020-12-15,"Facebook AI Research,New York University (NYU)",France; United States of America,Mixed or other,Biology,parameters,652400000,652400000,Confident,See Table 9,,,,Honeybee to mouse,5.1e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.pnas.org/doi/abs/10.1073/pnas.2016239118,CC BY 4.0 (Epoch AI)
model,VQGAN + CLIP,2020-12-17,Heidelberg University,Germany,Academia,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2012.09841,CC BY 4.0 (Epoch AI)
model,DensePhrases,2020-12-23,"Korea University,Princeton University",South Korea; United States of America,Academia,Language,parameters,,,Speculative,may be possible to estimate from batch size (8) and maximum memory of GPUs (96GB),,,,,2.09952e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2012.12624v3,CC BY 4.0 (Epoch AI)
model,CT-MoS (WT2),2020-12-25,"Google,National Tsing Hua University",Taiwan; United States of America,Mixed or other,Language,parameters,45000000,45000000,Confident,45M Table 2,,,,Below a fruit fly,5.4e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2012.13575,CC BY 4.0 (Epoch AI)
model,ERNIE-Doc (247M),2020-12-31,Baidu,China,Industry,Language,parameters,247000000,247000000,Confident,,,,,Fruit fly to honeybee,3.0302798e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2012.15688,CC BY 4.0 (Epoch AI)
model,CLIP (ResNet-50),2021-01-05,OpenAI,United States of America,Industry,"Multimodal,Vision,Language,Video",parameters,88600000,88600000,Likely,Image encoder ~ResNet-50 (from paper) 25.6M params  Text encoder ~Transformer (from paper) 63M params,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2103.00020,CC BY 4.0 (Epoch AI)
model,CLIP (ViT L/14@336px),2021-01-05,OpenAI,United States of America,Industry,"Multimodal,Vision,Language,Video",parameters,370000000,370000000,Confident,"Image encoder Vision Transformer Table 1 in https://arxiv.org/pdf/2010.11929.pdf Authors fine-tuned ViT L/14 at additional 336px resolution, hence the @336 (See ViT) 307M params  Text encoder ~Transformer (from paper) 63M params",,,,Fruit fly to honeybee,1.05e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2103.00020
https://huggingface.co/openai/clip-vit-large-patch14-336",CC BY 4.0 (Epoch AI)
model,DALL-E,2021-01-05,OpenAI,United States of America,Industry,Image generation,parameters,12000000000,12000000000,Likely,DALL·E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions,,,,Honeybee to mouse,4.7e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://openai.com/blog/dall-e/

https://arxiv.org/abs/2102.12092",CC BY 4.0 (Epoch AI)
model,BigSSL,2021-01-10,"Google,Apple",United States of America,Industry,Speech,parameters,8000000000,8000000000,,"""... we study the utility of large models, with the parameter count ranging from 600M to 8B...""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2109.13226,CC BY 4.0 (Epoch AI)
model,Switch,2021-01-11,Google,United States of America,Industry,Language,parameters,1571000000000,1571000000000,Confident,"""Combining expert, model and data parallelism, we design two large Switch Transformer models, one with 395 billion and 1.6 trillion parameters"" Table 9 gives more precise count of 1571B parameters",,,,Mouse to cat,8.22e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2101.03961,CC BY 4.0 (Epoch AI)
model,DeiT-B,2021-01-15,"Meta AI,Sorbonne University",France; United States of America,Mixed or other,Vision,parameters,86000000,86000000,Confident,(DeiT-B),,,,Fruit fly to honeybee,7.884e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2012.12877,CC BY 4.0 (Epoch AI)
model,DLWP,2021-02-09,"University of Washington,Microsoft Research",United States of America,Mixed or other,Earth science,parameters,2676376,2676376,Confident,Table 1,,,,Below a fruit fly,5.6845152e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2102.05107,CC BY 4.0 (Epoch AI)
model,top-down frozen classifier,2021-02-09,"University of Edinburgh,Toshiba Cambridge Research Laboratory",United Kingdom,Mixed or other,Speech,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2102.04697,CC BY 4.0 (Epoch AI)
model,MSA Transformer,2021-02-13,"Facebook AI Research,University of California (UC) Berkeley,New York University (NYU)",France; United States of America,Mixed or other,Biology,parameters,100000000,100000000,Likely,"""We train an MSA Transformer model with 100M parameters..."" ",,,,Fruit fly to honeybee,5.49e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.mlr.press/v139/rao21a/rao21a.pdf,CC BY 4.0 (Epoch AI)
model,Rational DQN Average,2021-02-18,TU Darmstadt,Germany,Academia,Games,parameters,1683456,1683456,,See figure 7,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2102.09407v3,CC BY 4.0 (Epoch AI)
model,SRU++ Large,2021-02-24,ASAPP,United States of America,Industry,Language,parameters,234000000,234000000,Confident,Table 5,,,,Fruit fly to honeybee,2.1173704e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2102.12459,CC BY 4.0 (Epoch AI)
model,Meta Pseudo Labels,2021-03-01,"Google Brain,Google AI",United States of America,Industry,Vision,parameters,480000000,480000000,Likely,Table 4  480M,,,,Honeybee to mouse,4.79e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2003.10580,CC BY 4.0 (Epoch AI)
model,Generative BST,2021-03-05,Facebook AI Research,France; United States of America,Industry,Language,parameters,9431810048,9431810048,Confident,The largest model is a transformer with 9.4B parameters (Table 2),,,,Honeybee to mouse,1.449e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2004.13637,CC BY 4.0 (Epoch AI)
model,M6-T,2021-03-05,Alibaba,China,Industry,"Multimodal,Language,Vision",parameters,1002700000000,1002700000000,Likely,Table 5. Note model is sparse MoE with 960 experts; not all parameters are activated on the forward pass.,yes,,,Mouse to cat,5.5e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2105.15082,CC BY 4.0 (Epoch AI)
model,Unicorn,2021-03-24,Allen Institute for AI,United States of America,Mixed or other,Language,parameters,11000000000,11000000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2103.13009,CC BY 4.0 (Epoch AI)
model,PLUG,2021-04-19,Alibaba,China,Industry,Language,parameters,27000000000,27000000000,Confident,,,,,Honeybee to mouse,3.5997696e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mp.weixin.qq.com/s/DAQomIkDa52Sef-ruyH5qg,CC BY 4.0 (Epoch AI)
model,ProtBERT-BFD,2021-05-04,"Technical University of Munich,Nvidia,Seoul National University,Google,Oak Ridge National Laboratory,Med AI Technology",China; Germany; South Korea; United States of America,Mixed or other,Biology,parameters,420000000,420000000,Confident,Table 2,,,,Honeybee to mouse,3.9e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.biorxiv.org/content/10.1101/2020.07.12.199554v3 or 
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9477085",CC BY 4.0 (Epoch AI)
model,ProtT5-XL-U50,2021-05-04,"Technical University of Munich,Med AI Technology,Nvidia,Oak Ridge National Laboratory,Google,Seoul National University",China; Germany; South Korea; United States of America,Mixed or other,Biology,parameters,3000000000,3000000000,Confident,Table 2,,,,Honeybee to mouse,1.8704498688e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2020.07.12.199554v3.full.pdf,CC BY 4.0 (Epoch AI)
model,ADM,2021-05-11,OpenAI,United States of America,Industry,Image generation,parameters,559000000,559000000,Confident,"Largest model is denoted ImageNet 512, has 559M parameters",,,,Honeybee to mouse,6.2e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2105.05233,CC BY 4.0 (Epoch AI)
model,MedBERT,2021-05-20,"Peng Cheng Laboratory,University of Texas at Houston",China; United States of America,Academia,Medicine,parameters,17000000,17000000,Likely,"17M from ""This is possibly due to the fact that the untrained Med-BERT is an over-parameterized model (around 17 million parameters) with a huge number of configurations, so it might overfit to the training data""",,,,Below a fruit fly,9.47e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41746-021-00455-y,CC BY 4.0 (Epoch AI)
model,CogView,2021-05-26,"Tsinghua University,Alibaba DAMO Academy",China,Mixed or other,Image generation,parameters,4000000000,4000000000,Likely,"""We propose CogView, a 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem.""",,,,Honeybee to mouse,2.68e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2105.13290,CC BY 4.0 (Epoch AI)
model,Transformer local-attention (NesT-B),2021-05-26,"Google Cloud,Google Research",United States of America,Industry,Vision,parameters,90100000,90100000,Confident,"Table A2, NesT-B is the largest size.",,,,Fruit fly to honeybee,2.40576e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2105.12723v4,CC BY 4.0 (Epoch AI)
model,ByT5-XXL,2021-05-28,"Google,Google Research",United States of America,Industry,Language,parameters,12900000000,12900000000,Likely,"12.9B, from Table 1",,,,Honeybee to mouse,8.1e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2105.13626,CC BY 4.0 (Epoch AI)
model,ViT-G/14,2021-06-08,"Google Brain,Google Research",United States of America,Industry,Vision,parameters,1843000000,1843000000,Confident,Table 2 of paper,,,,Honeybee to mouse,5.85e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.04560,CC BY 4.0 (Epoch AI)
model,CoAtNet,2021-06-09,"Google,Google Research,Google Brain",United States of America,Industry,Vision,parameters,2440000000,2440000000,Confident,,,,,Honeybee to mouse,4.27e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.04803v2,CC BY 4.0 (Epoch AI)
model,EMDR,2021-06-09,"Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),McGill University,DeepMind",Canada; United Kingdom,Mixed or other,Language,parameters,440000000,440000000,Confident,Table 2,,,,Honeybee to mouse,1.91e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.05346v2,CC BY 4.0 (Epoch AI)
model,DeBERTa,2021-06-10,Microsoft,United States of America,Industry,Language,parameters,1500000000,1500000000,Confident,"""...we scale up DeBERTa by training a larger version that consists of 48 Transform layers with 1.5 billion parameters""  Other versions are smaller and use a smaller pre-training dataset. These are distinguished in the paper (e.g. DeBERTa1.5B is the version of DeBERTa with 1.5 billion parameters).",,,,Honeybee to mouse,2.588e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2006.03654,CC BY 4.0 (Epoch AI)
model,ALIGN,2021-06-11,Google Research,United States of America,Industry,"Multimodal,Vision,Language",parameters,820000000,820000000,Confident,From author communication  480M (image tower) + 340 M (text tower),,,,Honeybee to mouse,2.598670000001e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2102.05918,CC BY 4.0 (Epoch AI)
model,Denoising Diffusion Probabilistic Models (LSUN Bedroom),2021-06-11,University of California (UC) Berkeley,United States of America,Academia,Vision,parameters,256000000,256000000,Confident,"Appendix B:  "" Our CIFAR10 model has 35.7 million parameters, and our LSUN and CelebA-HQ models have 114 million parameters. We also trained a larger variant of the LSUN Bedroom model with approximately 256 million parameters by increasing filter count.""",,,,Fruit fly to honeybee,7.840125000000001e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2006.11239,CC BY 4.0 (Epoch AI)
model,StyleGAN3-R,2021-06-21,"Nvidia,Aalto University",Finland; United States of America,Mixed or other,Image generation,parameters,1580000,1580000,Confident,,,,,Below a fruit fly,2.42784e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/2106.12423,CC BY 4.0 (Epoch AI)
model,StyleGAN3-T,2021-06-21,"Nvidia,Aalto University",Finland; United States of America,Mixed or other,Image generation,parameters,2230000,2230000,Confident,,,,,Below a fruit fly,1.70208e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.12423,CC BY 4.0 (Epoch AI)
model,EfficientNetV2-XL,2021-06-23,"Google,Google Brain",United States of America,Industry,Vision,parameters,208000000,208000000,Confident,"208M for XL version (Table 7, page 7)",,,,Fruit fly to honeybee,9.56e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2104.00298,CC BY 4.0 (Epoch AI)
model,Fold2Seq,2021-06-24,"IBM,Texas A&M",United States of America,Mixed or other,Biology,parameters,12427904,12427904,Speculative,"Three components, a sequence encoder, fold encoder, and sequence decoder. (1) and (3) have similar architectures, a sequence embedder + transformer block (the encoder uses a transformer encoder, the decoder uses a decoder block).   The fold encoder adds 6 residual conv blocks and has a 3d positional",,yes,,Below a fruit fly,1.4e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.13058,CC BY 4.0 (Epoch AI)
model,Adaptive Input Transformer + RD,2021-06-28,"Microsoft Research Asia,Soochow University",China; Taiwan,Mixed or other,Language,parameters,247000000,247000000,Likely,,,,,Fruit fly to honeybee,8.58e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.14448,CC BY 4.0 (Epoch AI)
model,ERNIE 3.0,2021-07-05,Baidu,China,Industry,Language,parameters,10000000000,10000000000,Confident,"""We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph.""",,,,Honeybee to mouse,2.25e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2107.02137,CC BY 4.0 (Epoch AI)
model,Codex,2021-07-07,OpenAI,United States of America,Industry,Language,parameters,12000000000,12000000000,Likely,"""With just a single sample, a 12B parameter Codex solves 28.8% of these problems, and a 300M parameter Codex solves 13.2% of these problems""",,,,Honeybee to mouse,7.344e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://openai.com/blog/openai-codex/
https://arxiv.org/abs/2107.03374",CC BY 4.0 (Epoch AI)
model,GOAT,2021-07-27,DeepMind,United Kingdom,Industry,Games,parameters,3472816,3472816,Speculative,estimate described here: https://docs.google.com/document/d/1S9xZyCeITDOs-P1W_-liNW0WgVN-OLsSudVrPXMaLqw/edit?usp=sharing,,yes,,Below a fruit fly,2.412e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play

https://arxiv.org/abs/2107.12808",CC BY 4.0 (Epoch AI)
model,HuBERT,2021-07-27,Facebook AI Research,France; United States of America,Industry,Speech,parameters,1000000000,1000000000,Speculative,"From abstract: ""Using a 1B parameter model, HuBERT shows up to 19% and 13% relative WER reduction on the more challenging dev-other and test-other evaluation subsets""",,yes,,Honeybee to mouse,5.54e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2106.07447,CC BY 4.0 (Epoch AI)
model,SEER,2021-07-29,"Facebook AI Research,INRIA",France; United States of America,Mixed or other,Vision,parameters,1300000000,1300000000,Confident,"From abstract: "" Our final SElf-supERvised (SEER) model, a RegNetY with 1.3B parameters...""",,,,Honeybee to mouse,1.8e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2103.01988,CC BY 4.0 (Epoch AI)
model,6-Act Tether,2021-08-03,"Facebook AI Research,Georgia Institute of Technology",France; United States of America,Mixed or other,Robotics,parameters,5000000,5000000,Confident,"""Agent parameter counts were all 5 − 6 million parameters, excluding parameters in auxiliary modules""",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://openaccess.thecvf.com/content/ICCV2021/html/Ye_Auxiliary_Tasks_and_Exploration_Enable_ObjectGoal_Navigation_ICCV_2021_paper.html
https://arxiv.org/abs/2104.04112",CC BY 4.0 (Epoch AI)
model,YOLOX-X,2021-08-06,Megvii Inc,China,Industry,Vision,parameters,99100000,99100000,Likely,"99.1M, table 3",,,,Fruit fly to honeybee,6.34275e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2107.08430,CC BY 4.0 (Epoch AI)
model,W2v-BERT,2021-08-07,"Google Brain,Massachusetts Institute of Technology (MIT)",United States of America,Mixed or other,Speech,parameters,1000000000,1000000000,Confident,1B for XXL model,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2108.06209v2,CC BY 4.0 (Epoch AI)
model,Jurassic-1-Jumbo,2021-08-11,AI21 Labs,Israel,Industry,Language,parameters,178000000000,178000000000,Confident,"""Jurassic-1 models come in two sizes, where the Jumbo version, at 178B parameters, is the largest and most sophisticated language model ever released for general use by developers.""",,,,Mouse to cat,3.7e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://uploads-ssl.webflow.com/60fd4503684b466578c0d307/61138924626a6981ee09caf6_jurassic_tech_paper.pdf,CC BY 4.0 (Epoch AI)
model,Zidong Taichu,2021-08-11,"Chinese Academy of Sciences,Wuhan AI Computing Center",China,Mixed or other,"Multimodal,Speech,Vision,Language",parameters,3200000000,3200000000,Confident,共32亿参数 translated as A total of 3.2 billion parameters ,,,,Honeybee to mouse,8.016e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://gitee.com/zidongtaichu/multi-modal-models,CC BY 4.0 (Epoch AI)
model,DNABERT,2021-08-15,Northeastern University,United States of America,Academia,Biology,parameters,110000000,110000000,Confident,"""We used the same model architecture as the BERT base, which consists of 12 Transformer layers with 768 hidden units and 12 attention heads in each layer, and the same parameter setting across all the four DNABERT models during pre-training""  Known to have 110 million parameters as reported in: http",,,,Fruit fly to honeybee,1.07e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://academic.oup.com/bioinformatics/article/37/15/2112/6128680,CC BY 4.0 (Epoch AI)
model,XLMR-XXL,2021-08-17,Facebook AI Research,France; United States of America,Industry,Language,parameters,10700000000,10700000000,Confident,"Section 2.1: "" ...XLM-RXXL (L= 48, H = 4096, A = 32, 10.7B params)""",,,,Honeybee to mouse,3.366e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2105.00572,CC BY 4.0 (Epoch AI)
model,FLAN 137B,2021-09-03,Google Research,United States of America,Industry,Language,parameters,137000000000,137000000000,Confident,"Abstract: ""We take a 137B parameter pretrained language model and instruction tune it on over 60 NLP datasets verbalized via natural language instruction templates. We evaluate this instruction-tuned model, which we call FLAN, on unseen task types.""  Many models seem to be using the same 137B base t",,,,Mouse to cat,2.047e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2109.01652,CC BY 4.0 (Epoch AI)
model,MEB,2021-09-04,Microsoft,United States of America,Industry,"Search,Language",parameters,135000000000,135000000000,Confident,See paper title,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.microsoft.com/en-us/research/blog/make-every-feature-binary-a-135b-parameter-sparse-neural-network-for-massively-improved-search-relevance/,CC BY 4.0 (Epoch AI)
model,PermuteFormer,2021-09-06,Peking University,China,Academia,Language,parameters,149697024,149697024,Speculative,"Parameterization appears to be similar to a vanilla transformer. 6 layers, hidden dimension of 512, feed forward dimension of 1024, 8 attention heads. This would imply 20,447,232 parameters without embedding weights, and 512*vocab_size embedding weights (assuming tied embedding and unembedding proje",,yes,,Fruit fly to honeybee,2.775e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2109.02377,CC BY 4.0 (Epoch AI)
model,HyperCLOVA 204B,2021-09-10,NAVER,South Korea,Industry,Language,parameters,204000000000,204000000000,Speculative,https://www.navercorp.com/navercorp_/ir/announce/2023/NAVER_CEO%20letter%20to%20shareholders_Aug%202023_Eng.pdf,,yes,,Mouse to cat,2.0000000001e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aibusiness.com/nlp/south-korea-s-naver-unveils-hyperscale-ai-platform-language-model-with-more-parameters-than-gpt-3,CC BY 4.0 (Epoch AI)
model,PLATO-XL,2021-09-20,Baidu,China,Industry,Language,parameters,11000000000,11000000000,Confident,,,,,Honeybee to mouse,9.9e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2109.09519,CC BY 4.0 (Epoch AI)
model,TrOCR,2021-09-21,"Beihang University,Microsoft Research Asia",China,Mixed or other,Vision,parameters,558000000,558000000,Confident,558M table 5,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2109.10282,CC BY 4.0 (Epoch AI)
model,Turing ULRv5,2021-09-28,Microsoft,United States of America,Industry,Language,parameters,2200000000,2200000000,Confident,2.2B,,,,Honeybee to mouse,2.8983951e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.microsoft.com/en-us/research/blog/microsoft-turing-universal-language-representation-model-t-ulrv5-tops-xtreme-leaderboard-and-trains-100x-faster/,CC BY 4.0 (Epoch AI)
model,AlphaFold-Multimer,2021-10-04,"Google DeepMind,DeepMind",United Kingdom; United States of America,Industry,Biology,parameters,,,Confident,"""Multiple changes to the AlphaFold system were made to adapt it to training on protein complexes, which are detailed below. Summarizing briefly, we [...] make various small adjustments to the structure losses and the model architecture."" [2. Methods]  Hence, this will have approximately the same amo",,,,,4.35e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2021.10.04.463034v1,CC BY 4.0 (Epoch AI)
model,Megatron-Turing NLG 530B,2021-10-11,"Microsoft,Nvidia",United States of America,Industry,Language,parameters,530000000000,530000000000,Confident,,,,,Mouse to cat,8.586e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2201.11990,CC BY 4.0 (Epoch AI)
model,Yuan 1.0,2021-10-12,Inspur,China,Industry,Language,parameters,245730000000,245730000000,Confident,"Table 2: Parameters of Yuan models. ""Parameters (billion)""",,,,Mouse to cat,3.5380000000001e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2110.04725,CC BY 4.0 (Epoch AI)
model,base LM+GNN+kNN,2021-10-17,"Shannon.AI,Nanjing University,Nanyang Technological University,Zhejiang University (ZJU)",China; Singapore,Mixed or other,Language,parameters,274000000,274000000,Confident,"274M (table 1)  ""We use the base version of deep Transformer language model with adaptive embeddings (Baevski & Auli, 2018) as our base LM. This model has 16 decoder layers. The dimensionality of word representations is 1,024, the number of multi-attention heads is 16, and the inner dimensionality o",,,,Fruit fly to honeybee,5.2587456e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2110.08743,CC BY 4.0 (Epoch AI)
model,Eve,2021-10-27,"Harvard Medical School,University of Oxford",United Kingdom; United States of America,Academia,Biology,parameters,15010300,15010300,Likely,"""The Bayesian VAE architecture in EVE is comprised of a symmetric 3-layer encoder & decoder architecture (with 2,000-1,000-300 and 300-1,000-2,000 units respectively) and a latent space of dimension 50 [...] We use a single set of parameters for the encoder (ϕp) and learn a fully-factorized gaussian",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-021-04043-8#change-history,CC BY 4.0 (Epoch AI)
model,EfficientZero,2021-10-30,"Tsinghua University,University of California (UC) Berkeley,Shanghai Qi Zhi institute",China; United States of America,Academia,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.00210,CC BY 4.0 (Epoch AI)
model,S4,2021-10-31,Stanford University,United States of America,Academia,Language,parameters,249000000,249000000,Likely,249M (Table 8),,,,Fruit fly to honeybee,7.8328627e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.00396,CC BY 4.0 (Epoch AI)
model,CodeT5-base,2021-11-01,"Salesforce,Nanyang Technological University",Singapore; United States of America,Mixed or other,Language,parameters,220000000,220000000,Likely,"""We build CodeT5 based on Huggingface’s T5 (Raffel et al., 2020) PyTorch implementation and employ two sizes of CodeT5-small (60M) and CodeT5-base (220M)""",,,,Fruit fly to honeybee,1.56e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aclanthology.org/2021.emnlp-main.685/,CC BY 4.0 (Epoch AI)
model,Projected GAN,2021-11-01,Heidelberg University,Germany,Academia,Image generation,parameters,,,Confident,Possibly calculable from Appendix Table 8,,,,,1.05e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://proceedings.neurips.cc/paper/2021/hash/9219adc5c42107c4911e249155320648-Abstract.html,CC BY 4.0 (Epoch AI)
model,Masked Autoencoders ViT-H,2021-11-11,Facebook AI Research,France; United States of America,Industry,Vision,parameters,632000000,632000000,Speculative,"Three models: ViT-B (86M), ViT-L (304M), ViT-H (632M)",,yes,,Honeybee to mouse,4.6e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.06377,CC BY 4.0 (Epoch AI)
model,ViT-G/14 (LiT),2021-11-15,Google Research,United States of America,Industry,Vision,parameters,3005000000,3005000000,Confident,Table 7,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.07991v3,CC BY 4.0 (Epoch AI)
model,AliceMind-MMU,2021-11-17,Alibaba,China,Industry,,parameters,,,Likely,"At least 5.415B becaues it uses 46 visual models which are either 110M (110M * 46 = 5.06B) or 345M  parameters each. It also uses StructureLM, which is 355M parameters, and some other relatively insignificant (in terms of parameter count) components.",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.08896,CC BY 4.0 (Epoch AI)
model,Swin Transformer V2 (SwinV2-G),2021-11-18,Microsoft Research Asia,China,Industry,"Vision,Video",parameters,3000000000,3000000000,Confident,,,,,Honeybee to mouse,1.1e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.09883v2,CC BY 4.0 (Epoch AI)
model,BASIC-L,2021-11-19,Google,United States of America,Industry,Vision,parameters,3070000000,3070000000,Likely,2.4B image model + 670M text model,,,,Honeybee to mouse,4.12e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.10050,CC BY 4.0 (Epoch AI)
model,Florence,2021-11-22,Microsoft,United States of America,Industry,Vision,parameters,893000000,893000000,Confident,"""Our Florence pretrained model has in total 893M parameters, including the language transformer with 256M parameters and the CoSwin-H transformer with 637M parameters.""",,,,Honeybee to mouse,4.831e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.11432v1,CC BY 4.0 (Epoch AI)
model,NÜWA,2021-11-24,"Microsoft Research,Peking University",China; United States of America,Mixed or other,"Multimodal,Vision,Image generation,Video,Language",parameters,870000000,870000000,Confident,Section 4.1,,,,Honeybee to mouse,7.24598784e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2111.12417,CC BY 4.0 (Epoch AI)
model,T-NLRv5 XXL,2021-12-03,Microsoft,United States of America,Industry,Language,parameters,5400000000,5400000000,Confident,Table 1 of the blogpost,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.marketscreener.com/quote/stock/MICROSOFT-CORPORATION-4835/news/Microsoft-Turing-NLRv5-achieves-new-performance-milestones-37207301/
https://www.microsoft.com/en-us/research/blog/efficiently-and-effectively-scaling-up-language-model-pretraining-for-best-language-representation-model-on-glue-and-superglue/",CC BY 4.0 (Epoch AI)
model,Student of Games,2021-12-06,DeepMind,United Kingdom,Industry,Games,parameters,,,Speculative,,,,,,3.6679273004682866e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.03178,CC BY 4.0 (Epoch AI)
model,Gopher (280B),2021-12-08,DeepMind,United Kingdom,Industry,Language,parameters,280000000000,280000000000,Confident,"Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world. In this paper, we present an analysis of Transformer-based language model performance across a wide range of model scal",,,,Mouse to cat,6.31e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.11446,CC BY 4.0 (Epoch AI)
model,GLaM,2021-12-13,Google,United States of America,Industry,Language,parameters,1200000000000,1200000000000,Confident,1.2 trillion parameters,,,,Mouse to cat,3.6363112434e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.06905,CC BY 4.0 (Epoch AI)
model,LongT5,2021-12-15,Google Research,United States of America,Industry,Language,parameters,3000000000,3000000000,Confident,3B from section 4.1,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.07916,CC BY 4.0 (Epoch AI)
model,Contriever,2021-12-16,"Meta AI,University College London (UCL),PSL University,Université Grenoble Alpes",France; United Kingdom; United States of America,Mixed or other,Language,parameters,110000000,110000000,Likely,"Based on BERT base, which had 110m params.  ""We initialize the network with the publicly available BERT base uncased model.""",,,,Fruit fly to honeybee,1.57e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.09118,CC BY 4.0 (Epoch AI)
model,LDM-1.45B,2021-12-20,"Heidelberg University,Runway",Germany; United States of America,Mixed or other,Image generation,parameters,1450000000,1450000000,Confident,1.45B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.10752,CC BY 4.0 (Epoch AI)
model,XGLM-7.5B,2021-12-20,"Meta AI,Facebook AI Research",France; United States of America,Industry,Language,parameters,7500000000,7500000000,Confident,"""Our largest model with 7.5 billion parameters sets new state of the art""",,,,Honeybee to mouse,2.25e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.10668,CC BY 4.0 (Epoch AI)
model,ERNIE 3.0 Titan,2021-12-23,"Baidu,Peng Cheng Laboratory",China,Mixed or other,Language,parameters,260000000000,260000000000,Confident,"""[We] developed... distributed training technology, including fine-grained parallelism, heterogeneous hardware-aware training, and fault tolerance mechanism to train the 260B model on both Nvidia V100 GPU and Ascend 910 NPU clusters."" See also: https://twitter.com/BaiduResearch/status/14686339772422",,,,Mouse to cat,1.0421e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.12731,CC BY 4.0 (Epoch AI)
model,ERNIE-ViLG,2021-12-31,Baidu,China,Industry,"Multimodal,Image generation,Vision,Language",parameters,10000000000,10000000000,,"""To explore the landscape of large-scale pre-training for bidirectional text-image generation, we pre-train a 10-billion parameter model on a large-scale dataset of 145 million high-quality Chinese image-text pairs.""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.15283,CC BY 4.0 (Epoch AI)
model,Detic,2022-01-07,"Meta AI,University of Texas at Austin",United States of America,Mixed or other,Vision,parameters,88000000,88000000,Speculative,"from https://github.com/microsoft/Swin-Transformer Swin-B have 88M,  from page 8 :  'Training our ResNet50 model takes ∼ 22 hours on 8 V100 GPUs. The large 21K Swin-B model trains in ∼ 24 hours on 32 GPUs.'",,yes,,Fruit fly to honeybee,2.34399744e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2201.02605,CC BY 4.0 (Epoch AI)
model,data2vec (language),2022-01-20,Meta AI,United States of America,Industry,Language,parameters,705134592,705134592,Likely,"Section 4: ""We experiment with two model sizes: data2vec Base and data2vec Large, containing either L = 12 or L = 24 Trans- former blocks with H = 768 or H = 1024 hidden dimen- sion (with 4 × H feed-forward inner-dimension)""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.facebook.com/research/data2vec-a-general-framework-for-self-supervised-learning-in-speech-vision-and-language/,CC BY 4.0 (Epoch AI)
model,data2vec (speech),2022-01-20,Meta AI,United States of America,Industry,Speech,parameters,705134592,705134592,,"Section 4: ""We experiment with two model sizes: data2vec Base and data2vec Large, containing either L = 12 or L = 24 Trans- former blocks with H = 768 or H = 1024 hidden dimen- sion (with 4 × H feed-forward inner-dimension)"" ",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.facebook.com/research/data2vec-a-general-framework-for-self-supervised-learning-in-speech-vision-and-language/,CC BY 4.0 (Epoch AI)
model,data2vec (vision),2022-01-20,Meta AI,United States of America,Industry,Vision,parameters,705134592,705134592,,"Section 4: ""We experiment with two model sizes: data2vec Base and data2vec Large, containing either L = 12 or L = 24 Trans- former blocks with H = 768 or H = 1024 hidden dimen- sion (with 4 × H feed-forward inner-dimension)"" ",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.facebook.com/research/data2vec-a-general-framework-for-self-supervised-learning-in-speech-vision-and-language/,CC BY 4.0 (Epoch AI)
model,AbLang (heavy sequences),2022-01-22,University of Oxford,United Kingdom,Academia,Biology,parameters,355000000,355000000,Confident,"""The hyperparameters were selected to be similar to those used in the RoBERTa paper (Liu et al., 2019).""  Liu et al., 2019 link: https://arxiv.org/pdf/1907.11692.pdf ""We begin by training RoBERTa following the BERTLARGE architecture (L = 24, H = 1024, A = 16, 355M parameters)""",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://academic.oup.com/bioinformaticsadvances/article/2/1/vbac046/6609807,CC BY 4.0 (Epoch AI)
model,OntoProtein,2022-01-23,Zhejiang University (ZJU),China,Academia,"Biology,Language",parameters,420000000,420000000,Likely,"""For the protein encoder, we use the pre-trained ProtBert from Elnaggar et al. (2020).""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openreview.net/pdf?id=yfe1VMYAXa4,CC BY 4.0 (Epoch AI)
model,InstructGPT 1.3B,2022-01-27,OpenAI,United States of America,Industry,Language,parameters,1300000000,1300000000,Confident,"""We train three model sizes (1.3B, 6B, and 175B parameters)""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.02155,CC BY 4.0 (Epoch AI)
model,InstructGPT 175B,2022-01-27,OpenAI,United States of America,Industry,Language,parameters,175000000000,175000000000,Confident,"""We train three model sizes (1.3B, 6B, and 175B parameters)""",,,,Mouse to cat,3.19181e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/2203.02155,CC BY 4.0 (Epoch AI)
model,InstructGPT 6B,2022-01-27,OpenAI,United States of America,Industry,Language,parameters,6000000000,6000000000,Confident,"""We train three model sizes (1.3B, 6B, and 175B parameters)""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.02155,CC BY 4.0 (Epoch AI)
model,AlphaCode,2022-02-02,DeepMind,United Kingdom,Industry,Language,parameters,41100000000,41100000000,Confident,41.1B. Table 3,,,,Honeybee to mouse,2.38010000000001e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.07814,CC BY 4.0 (Epoch AI)
model,RETRO-7B,2022-02-07,DeepMind,United Kingdom,Industry,Language,parameters,7500000000,7500000000,Confident,"""Retro provides a constant gain for models ranging from 150M to 7B parameters, and Retro can be improved at evaluation time by increasing the database size and the number of retrieved neighbours. """,,,,Honeybee to mouse,1.68e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.04426,CC BY 4.0 (Epoch AI)
model,MaskGIT (ImageNet),2022-02-08,Google Research,United States of America,Industry,Image generation,parameters,227000000,227000000,Confident,227M Table 1,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2202.04200,CC BY 4.0 (Epoch AI)
model,GPT-NeoX-20B,2022-02-09,EleutherAI,United States of America,Mixed or other,Language,parameters,20000000000,20000000000,Confident,,,,,Honeybee to mouse,9.31627008e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2204.06745,CC BY 4.0 (Epoch AI)
model,LaMDA,2022-02-10,Google,United States of America,Industry,Language,parameters,137000000000,137000000000,Confident,"""LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters""",,,,Mouse to cat,3.55e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2201.08239,CC BY 4.0 (Epoch AI)
model,ProteinBERT,2022-02-10,"Hebrew University of Jerusalem,Ben-Gurion University of the Negev,Deep Trading",Israel; United States of America,Mixed or other,Biology,parameters,16000000,16000000,Confident,"""Altogether, it includes ∼16M trainable parameters, making it substantially smaller than other protein language models""",,,,Below a fruit fly,6.5e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://academic.oup.com/bioinformatics/article/38/8/2102/6502274,CC BY 4.0 (Epoch AI)
model,Midjourney V1,2022-02-15,Midjourney,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,,CC BY 4.0 (Epoch AI)
model,ST-MoE,2022-02-17,"Google,Google Brain,Google Research",United States of America,Industry,Language,parameters,269000000000,269000000000,Likely,269B. it's called ST-MoE-32B because it's equivalent to a 32B dense model.,yes,,,Mouse to cat,2.9e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2202.08906v2,CC BY 4.0 (Epoch AI)
model,FourCastNet,2022-02-22,"Nvidia,NERSC, Lawrence Berkeley National Laboratory,University of Michigan,Rice University,California Institute of Technology,Purdue University",United States of America,Mixed or other,Earth science,parameters,,,Confident,,,,,,3.4504704e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2202.11214,CC BY 4.0 (Epoch AI)
model,PolyCoder,2022-02-26,Carnegie Mellon University (CMU),United States of America,Academia,Language,parameters,2700000000,2700000000,Likely,2.7B for largest model,,,,Honeybee to mouse,1.1e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2202.13169,CC BY 4.0 (Epoch AI)
model,DeepNet,2022-03-01,Microsoft Research,United States of America,Industry,Language,parameters,3200000000,3200000000,Confident,"""Remarkably, on a multilingual benchmark with 7,482 translation directions, our 200-layer model with 3.2B parameters significantly outperforms the 48-layer state-of-the-art model with 12B parameters by 5 BLEU points, which indicates a promising scaling direction""  EDIT 05/05/2022: The 12B model was ",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.00555,CC BY 4.0 (Epoch AI)
model,Statement Curriculum Learning,2022-03-02,OpenAI,United States of America,Industry,"Language,Mathematics",parameters,774000000,774000000,Likely,,,,,Honeybee to mouse,1.7901648e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2202.01344,CC BY 4.0 (Epoch AI)
model,MegaSyn,2022-03-07,Collaborations Pharmaceuticals,United States of America,Industry,Medicine,parameters,,,Unknown,"model details here: https://chemrxiv.org/engage/chemrxiv/article-details/61551803d1fc335b7cf8fd45  ""The variational autoencoder utilizes an encoder-decoder architecture to map chemical space into a latent vector 34. The encoder is composed of 3 LSTM layers of 512 units each followed by a linear laye",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9544280/,CC BY 4.0 (Epoch AI)
model,ViT-G (model soup),2022-03-10,"University of Washington,Columbia University,Google,Meta AI,Tel Aviv University",Israel; United States of America,Mixed or other,Vision,parameters,1843000000,1843000000,Confident,This is from the original ViT-G paper,,,,Honeybee to mouse,3.4e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.05482v3,CC BY 4.0 (Epoch AI)
model,GPT-3.5 (davinci-002),2022-03-15,OpenAI,United States of America,Industry,Language,parameters,,,Speculative,"Parameter count may be 175B based on OpenAI's statements that text-davinci-003 is in the GPT-3.5 series of models. It was also stated to be 175B in the Microsoft CODEFUSION paper, but the paper was reportedly retracted because the authors did not know the parameter count.",,,,,2.578e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://platform.openai.com/docs/models/gpt-3-5,CC BY 4.0 (Epoch AI)
model,"Segatron-XL large, M=384 + HCP",2022-03-21,"Microsoft Research,University of Waterloo",Canada; United States of America,Mixed or other,Language,parameters,256999999,256999999,Confident,"257M (Table 1)  18 layers, 16 heads, hidden size 1024 batch size 128.",,,,Fruit fly to honeybee,2.65e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.10692,CC BY 4.0 (Epoch AI)
model,Make-A-Scene,2022-03-24,Meta AI,United States of America,Industry,Image generation,parameters,4000000000,4000000000,Likely,"""Experiments were performed with a 4 billion parameter transformer, generating a sequence of 256 text tokens, 256 scene tokens, and 1024 image tokens, that are then decoded into an image with a resolution of 256 × 256 or 512 × 512 pixels (depending on the model of choice)""",,,,Honeybee to mouse,6.4172851e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.13131,CC BY 4.0 (Epoch AI)
model,Chinchilla,2022-03-29,DeepMind,United Kingdom,Industry,Language,parameters,70000000000,70000000000,Confident,"""We test this hypothesis by training a predicted compute-optimal model, \chinchilla, that uses the same compute budget as \gopher but with 70B parameters and 4× more more data. \chinchilla uniformly and significantly outperforms \Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NL",,,,Honeybee to mouse,5.76e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.15556,CC BY 4.0 (Epoch AI)
model,PaLM (540B),2022-04-04,Google Research,United States of America,Industry,Language,parameters,540350000000,540350000000,Confident,"""To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model (PaLM).""",yes,,,Mouse to cat,2.5272e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2204.02311,CC BY 4.0 (Epoch AI)
model,DALL·E 2,2022-04-06,OpenAI,United States of America,Industry,Image generation,parameters,3500000000,3500000000,Speculative,"""Our decoder architecture is the 3.5 billion parameter GLIDE model""",,yes,,Honeybee to mouse,3.3695784e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/papers/dall-e-2.pdf,CC BY 4.0 (Epoch AI)
model,BERT-RBP,2022-04-07,Waseda University,Japan,Academia,Biology,parameters,110000000,110000000,Confident,"Base model is BERT base (110M parameters), pre-trained on human reference genome (DNABert: https://academic.oup.com/bioinformatics/article/37/15/2112/6128680)",,,,Fruit fly to honeybee,1.4e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://academic.oup.com/bioinformaticsadvances/article/2/1/vbac023/6564689,CC BY 4.0 (Epoch AI)
model,Stable Diffusion (LDM-KL-8-G),2022-04-13,"Runway,Ludwig Maximilian University of Munich,Heidelberg University",Germany; United States of America,Mixed or other,Image generation,parameters,1450000000,1450000000,Confident,See Table 2,,,,Honeybee to mouse,5.0000000000000004e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.10752,CC BY 4.0 (Epoch AI)
model,Sparse all-MLP,2022-04-14,Meta AI,United States of America,Industry,Language,parameters,9410000000,9410000000,Confident,"Table 2: ""In Section 4.4, we run our large model (9.41B parameters)""",,,,Honeybee to mouse,5.32224e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2203.06850,CC BY 4.0 (Epoch AI)
model,XMC-GAN,2022-04-14,Google Research,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2101.04702,CC BY 4.0 (Epoch AI)
model,Flamingo,2022-04-29,DeepMind,United Kingdom,Industry,"Multimodal,Vision,Language,Video",parameters,80000000000,80000000000,Confident,"""We obtain three models, Flamingo-3B, Flamingo-9B and Flamingo-80B""  "" The Flamingo-80B model builds on top of the frozen Chinchilla 70B language model [42]. Starting from the very first layer and before every seventh transformer blocks, we add a GATED XATTN-DENSE layer attending to the visual input",,,,Honeybee to mouse,2.18972000000001e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2204.14198,CC BY 4.0 (Epoch AI)
model,OPT-175B,2022-05-02,Meta AI,United States of America,Industry,Language,parameters,175000000000,175000000000,Confident,"""In line with Meta AI’s commitment to open science, we are sharing Open Pretrained Transformer (OPT-175B), a language model with 175 billion parameters trained on publicly available data sets""",,,,Mouse to cat,4.3e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.01068,CC BY 4.0 (Epoch AI)
model,DeBERTaV3large + KEAR,2022-05-04,Microsoft,United States of America,Industry,Language,parameters,418000000,418000000,Confident,"DeBERTaV3-large had 418M params, per Table 2",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2112.03254v3,CC BY 4.0 (Epoch AI)
model,UL2,2022-05-10,"Google Research,Google Brain",United States of America,Industry,Language,parameters,20000000000,20000000000,Confident,Taken from Directory of LLMs,,,,Honeybee to mouse,1.2e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.05131v1,CC BY 4.0 (Epoch AI)
model,Gato,2022-05-12,DeepMind,United Kingdom,Industry,"Multimodal,Robotics,Games,Language",parameters,1180000000,1180000000,Confident,"""This section focuses on in-simulation evaluation. Figure 10 compares the full 1.18B parameter Gato"" p.10",,,,Honeybee to mouse,4.02e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.06175,CC BY 4.0 (Epoch AI)
model,SimCSE,2022-05-18,"Princeton University,Tsinghua University",China; United States of America,Academia,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2104.08821,CC BY 4.0 (Epoch AI)
model,Imagen,2022-05-23,Google Brain,United States of America,Industry,Image generation,parameters,7762000000,7762000000,Likely,"2B 64x64 generation model, 600M 64->256 super-resolution model, 400M 256->1024 super-resolution model Uses encodings from a frozen T5-XXL, which should be included in total parameter count. Loading the model directly, there are 4,762,310,656 parameters in the encoder. 2B + 4.762B + 600M + 400M = 7.7",,,,Honeybee to mouse,1.46e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.11487,CC BY 4.0 (Epoch AI)
model,GPT-2 Medium (FlashAttention),2022-05-27,"Stanford University,University at Buffalo",United States of America,Academia,Language,parameters,355000000,355000000,Confident,"GPT-2 Medium from here, aka GPT-2 355M: https://huggingface.co/openai-community/gpt2-medium",,,,Fruit fly to honeybee,8.9280922e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.14135,CC BY 4.0 (Epoch AI)
model,Tranception,2022-05-27,"University of Oxford,Harvard Medical School,Cohere",Canada; United Kingdom; United States of America,Mixed or other,Biology,parameters,700000000,700000000,Confident,"""Our largest transformer model, Tranception L, has 700M parameters and is trained on UniRef100 (Suzek et al., 2014)""",,,,Honeybee to mouse,7.24e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.13760,CC BY 4.0 (Epoch AI)
model,CogVideo,2022-05-29,"Tsinghua University,Beijing Academy of Artificial Intelligence / BAAI",China,Academia,Video,parameters,9400000000,9400000000,Speculative,,,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.15868,CC BY 4.0 (Epoch AI)
model,Diffusion-GAN,2022-06-05,"UT Austin,Microsoft",United States of America,Mixed or other,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2206.02262v4,CC BY 4.0 (Epoch AI)
model,DITTO,2022-06-06,"Tsinghua University,Apple,Westlake University,Chinese University of Hong Kong (CUHK)",China; Hong Kong; United States of America,Mixed or other,Language,parameters,750000000,750000000,Confident,"""We train a Transformer model (750M parameters, similar to GPT-2 Large)""  ""Specifically, we use a 16-layer Transformer with 8 attention heads, hidden size 1024 and fully-connected dimension 4096.""",,,,Honeybee to mouse,3.31776e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2206.02369,CC BY 4.0 (Epoch AI)
model,MetaLM,2022-06-13,Microsoft Research,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Speculative,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2206.06336v1,CC BY 4.0 (Epoch AI)
model,CoCa,2022-06-14,Google Research,United States of America,Industry,Vision,parameters,2100000000,2100000000,Confident,"""Our largest CoCa model (""CoCa"" in short) follows the ViT-giant setup in [21] with 1B-parameters in the image encoder and 2.1B-parameters altogether with the text decoder""",,,,Honeybee to mouse,7.3e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2205.01917v2,CC BY 4.0 (Epoch AI)
model,Parti,2022-06-22,Google Research,United States of America,Industry,Image generation,parameters,20000000000,20000000000,Confident,"Abstract: ""we achieve consistent quality improvements by scaling the encoder-decoder Transformer model up to 20B parameters""",,,,Honeybee to mouse,5.09607936e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2206.10789v1,CC BY 4.0 (Epoch AI)
model,ProGen2-xlarge,2022-06-27,"Salesforce Research,Columbia University,Johns Hopkins University",United States of America,Mixed or other,Biology,parameters,6400000000,6400000000,Confident,"""We introduce a suite of protein language models, named ProGen2, that are scaled up to 6.4B parameters""",,,,Honeybee to mouse,1.35e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2206.13517,CC BY 4.0 (Epoch AI)
model,Minerva (540B),2022-06-29,Google,United States of America,Industry,Language,parameters,540350000000,540350000000,Confident,"""To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model (PaLM).""  Our approach is to start with the PaLM pretrained decoder-only transformer language models Cho",yes,,,Mouse to cat,2.7415e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2206.14858,CC BY 4.0 (Epoch AI)
model,CodeT5-large,2022-07-05,Salesforce,United States of America,Industry,Language,parameters,770000000,770000000,Likely,"""We pretrain a CodeT5-large model (770M) from scratch following T5-large’s architecture""",,,,Honeybee to mouse,2.72e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2207.01780,CC BY 4.0 (Epoch AI)
model,NLLB,2022-07-06,Meta AI,United States of America,Industry,Language,parameters,54500000000,54500000000,Confident,"Section 8.2.4: ""The model has a total of 54.5B parameters and FLOPs similar to that of a 3.3B dense model""",,,,Honeybee to mouse,1.751113728e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.facebook.com/publications/no-language-left-behind/,CC BY 4.0 (Epoch AI)
model,BLOOM-176B,2022-07-11,"Hugging Face,BigScience",France; United States of America,Mixed or other,Language,parameters,176247271424,176247271424,Confident,"See ""Technical Specifications"" on Hugging Face: https://huggingface.co/bigscience/bloom",,,,Mouse to cat,3.65664e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.05100,CC BY 4.0 (Epoch AI)
model,ESM2-15B,2022-07-21,"Meta AI,New York University (NYU),Stanford University,Massachusetts Institute of Technology (MIT)",United States of America,Mixed or other,Biology,parameters,15000000000,15000000000,Confident,"""we train models up to 15B parameters""",,,,Honeybee to mouse,7.35000000001e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.science.org/doi/abs/10.1126/science.ade2574
https://www.biorxiv.org/content/10.1101/2022.07.20.500902v2",CC BY 4.0 (Epoch AI)
model,OmegaPLM,2022-07-22,"Massachusetts Institute of Technology (MIT),Westlake University",China; United States of America,Academia,Biology,parameters,670000000,670000000,Confident,"""Our model contains 66 layers with around 670 million parameters without sharing parameters, which doubles the layer count of ESM-1b but roughly retains the parameter count.""",,,,Honeybee to mouse,1.03514112e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2022.07.21.500999v1,CC BY 4.0 (Epoch AI)
model,AlexaTM 20B,2022-08-02,Amazon,United States of America,Industry,Language,parameters,19750000000,19750000000,Confident,See Table 1 on p.3 of the paper,,,,Honeybee to mouse,2.04374016e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2208.01448,CC BY 4.0 (Epoch AI)
model,GLM-130B,2022-08-04,Tsinghua University,China,Academia,Language,parameters,130000000000,130000000000,Confident,Dense model,,,,Mouse to cat,3.5490054945e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.02414,CC BY 4.0 (Epoch AI)
model,BlenderBot 3,2022-08-10,"McGill University,Meta AI,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms)",Canada; United States of America,Mixed or other,Language,parameters,175000000000,175000000000,Likely,,,,,Mouse to cat,4.3e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2208.03188, https://github.com/facebookresearch/ParlAI/blob/main/parlai/zoo/bb3/model_card.md

training code: https://parl.ai/projects/bb3/ ",CC BY 4.0 (Epoch AI)
model,BEIT-3,2022-08-22,Microsoft,United States of America,Industry,"Multimodal,Vision,Language",parameters,1900000000,1900000000,Likely,1.9B from Table 2,,,,Honeybee to mouse,7e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2208.10442,CC BY 4.0 (Epoch AI)
model,PaLI,2022-09-14,Google,United States of America,Industry,"Language,Vision,Multimodal",parameters,16900000000,16900000000,Likely,"3.9b Image Encoder,  14b Multimodal Encoder-Decoder",,,,Honeybee to mouse,1.69e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2209.06794v4,CC BY 4.0 (Epoch AI)
model,SauTech,2022-09-14,"Saudi Data and Artificial Intelligence Authority,Saudi Company for Artificial Intelligence",Saudi Arabia,Mixed or other,"Speech,Audio",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://globalaisummit.org/en/News/Pages/NewsDetails.aspx?NewsId=73
https://saudigazette.com.sa/article/624984/SAUDI-ARABIA/SDAIA-SCAI-unveil-SauTech-speech-to-text-software",CC BY 4.0 (Epoch AI)
model,Whisper,2022-09-21,OpenAI,United States of America,Industry,Speech,parameters,1550000000,1550000000,Likely,Table 1,,,,Honeybee to mouse,4.2072663e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://cdn.openai.com/papers/whisper.pdf

https://arxiv.org/abs/2212.04356",CC BY 4.0 (Epoch AI)
model,Make-A-Video,2022-09-29,Meta AI,United States of America,Industry,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2209.14792,CC BY 4.0 (Epoch AI)
model,DiffDock,2022-10-04,Massachusetts Institute of Technology (MIT),United States of America,Academia,Biology,parameters,20240000,20240000,Likely,"""For determining the hyperparameters of DIFFDOCK’s score model, we trained smaller models (3.97 million parameters) that fit into 48GB of GPU RAM before scaling it up to the final model (20.24 million parameters) that was trained on four 48GB GPUs""  There's a separate 4.77M ""confidence model"" that h",,,,Below a fruit fly,7.2e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2210.01776, https://docs.nvidia.com/bionemo-framework/latest/models/diffdock.html",CC BY 4.0 (Epoch AI)
model,AlphaTensor,2022-10-05,DeepMind,United Kingdom,Industry,"Other,Games,Mathematics",parameters,,,Confident,,,,,,7.1414784e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-022-05172-4,CC BY 4.0 (Epoch AI)
model,Phenaki,2022-10-05,"University College London (UCL),University of Michigan,Google Brain",United Kingdom; United States of America,Mixed or other,Video,parameters,1800000000,1800000000,,"Unless specified otherwise, we train a 1.8B parameter Phenaki model on a corpus of ∼15M textvideo pairs at 8 FPS mixed with ∼50M text-images plus ∼400M pairs of LAION-400M [41] (more details in Appendix B.3). The model used in the visualisations in this paper was trained for 1 million steps at a bat",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.02399,CC BY 4.0 (Epoch AI)
model,Diplodocus,2022-10-11,"Meta AI,Massachusetts Institute of Technology (MIT)",United States of America,Mixed or other,Games,parameters,,,Unknown,may be estimated from https://github.com/facebookresearch/diplomacy_cicero?tab=readme-ov-file,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.05492,CC BY 4.0 (Epoch AI)
model,GenSLM,2022-10-11,"University of Chicago,Nvidia,Harvard University,Cerebras Systems,Technical University of Munich,California Institute of Technology",Germany; United States of America,Mixed or other,Biology,parameters,25000000000,25000000000,Confident,See Table 3,,,,Honeybee to mouse,1.42e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/biorxiv/early/2022/10/11/2022.10.10.511571.full.pdf,CC BY 4.0 (Epoch AI)
model,Flan-PaLM 540B,2022-10-20,Google,United States of America,Industry,Language,parameters,540000000000,540000000000,Confident,540B,,,,Mouse to cat,2.540000000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.11416,CC BY 4.0 (Epoch AI)
model,LMSI-Palm,2022-10-20,"Google,University of Illinois Urbana-Champaign (UIUC)",United States of America,Mixed or other,Language,parameters,540000000000,540000000000,Confident,540B,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.11610,CC BY 4.0 (Epoch AI)
model,U-PaLM (540B),2022-10-20,Google,United States of America,Industry,Language,parameters,540000000000,540000000000,Confident,,,,,Mouse to cat,2.53e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.11399,CC BY 4.0 (Epoch AI)
model,EnCodec,2022-10-24,Meta AI,United States of America,Industry,Audio,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2210.13438, ",CC BY 4.0 (Epoch AI)
model,eDiff-I,2022-11-02,Nvidia,United States of America,Industry,Image generation,parameters,9100000000,9100000000,Likely,"9.1B for config D, Table 1",,,,Honeybee to mouse,5.46e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.01324,CC BY 4.0 (Epoch AI)
model,Mogrifier RLSTM (WT2),2022-11-03,DeepMind,United Kingdom,Industry,Language,parameters,35000000,35000000,Confident,Table 1,,,,Below a fruit fly,1.4e+17,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.01848,CC BY 4.0 (Epoch AI)
model,mT0-13B,2022-11-03,"Hugging Face,BigScience",France; United States of America,Mixed or other,Language,parameters,13000000000,13000000000,Confident,13B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.01786,CC BY 4.0 (Epoch AI)
model,InternImage,2022-11-10,"Shanghai AI Lab,Tsinghua University,Nanjing University,SenseTime,Chinese University of Hong Kong (CUHK)",China; Hong Kong,Mixed or other,Vision,parameters,1080000000,1080000000,Confident,"1.08B, table 1",,,,Honeybee to mouse,2.408e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.05778,CC BY 4.0 (Epoch AI)
model,AltCLIP_M9,2022-11-12,Beijing Academy of Artificial Intelligence / BAAI,China,Academia,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.06679,CC BY 4.0 (Epoch AI)
model,EVA-01,2022-11-14,"Beijing Academy of Artificial Intelligence / BAAI,Huazhong University of Science and Technology,Zhejiang University (ZJU),Beijing Institute of Technology",China,Academia,Vision,parameters,1011000000,1011000000,Confident,1011M from table 3,,,,Honeybee to mouse,1.501e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.07636,CC BY 4.0 (Epoch AI)
model,Galactica,2022-11-16,Meta AI,United States of America,Industry,"Language,Biology",parameters,120000000000,120000000000,Likely,"""The largest 120B model we train runs on a single NVIDIA A100 node""",,,,Mouse to cat,3.24e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.09085,CC BY 4.0 (Epoch AI)
model,Fusion in Encoder,2022-11-18,Samsung,South Korea,Industry,Language,parameters,330000000,330000000,Likely,330M,,,,Fruit fly to honeybee,1.3e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.10147,CC BY 4.0 (Epoch AI)
model,AR-LDM,2022-11-20,"Alibaba,University of Waterloo,Vector Institute",Canada; China,Mixed or other,Image generation,parameters,1500000000,1500000000,Confident,Table 1,,,,Honeybee to mouse,5.1e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.10950,CC BY 4.0 (Epoch AI)
model,CICERO,2022-11-22,Meta AI,United States of America,Industry,Games,parameters,,,Unknown,"""We took R2C2 (22) as our base model – a 2.7B parameter Transformer-based (23) encoder-decoder model pre-trained on text from the Internet using a BART de-noising objective (24).""",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.science.org/doi/10.1126/science.ade9097,CC BY 4.0 (Epoch AI)
model,ALM 1.0,2022-11-28,Beijing Academy of Artificial Intelligence / BAAI,China,Academia,Language,parameters,335000000,335000000,Speculative,335M parameters: https://github.com/FlagAI-Open/FlagAI/blob/master/examples/ALM/README.md,,yes,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://github.com/FlagAI-Open/FlagAI/blob/master/examples/ALM/README.md,CC BY 4.0 (Epoch AI)
model,DiT-XL/2 + Discriminator Guidance,2022-11-28,"Korea Advanced Institute of Science and Technology (KAIST),NAVER",South Korea,Mixed or other,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.17091v4,CC BY 4.0 (Epoch AI)
model,Discriminator Guidance,2022-11-28,"Korea Advanced Institute of Science and Technology (KAIST),NAVER",South Korea,Mixed or other,Image generation,parameters,,,Confident,,,,,,2.1570000001e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2211.17091v4,CC BY 4.0 (Epoch AI)
model,DeepNash,2022-12-01,DeepMind,United Kingdom,Industry,Games,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.science.org/stoken/author-tokens/ST-887/full,CC BY 4.0 (Epoch AI)
model,Vega v2,2022-12-04,"Wuhan University,JD Explore Academy,Shanghai AI Lab,Nanyang Technological University,Washington University in St Louis,Chongqing University of Posts and Telecommunications,University of Sydney",Australia; China; Singapore; United States of America,Academia,Language,parameters,6000000000,6000000000,Confident,,,,,Honeybee to mouse,7.76e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2212.01853,CC BY 4.0 (Epoch AI)
model,TranceptEve,2022-12-10,"University of Oxford,Harvard Medical School",United Kingdom; United States of America,Academia,Biology,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2022.12.07.519495v1,CC BY 4.0 (Epoch AI)
model,RT-1,2022-12-13,Google,United States of America,Industry,Robotics,parameters,35000000,35000000,Confident,"""we also limit the size of the model compared to the original publication, which was 1.2B parameters (resulting in on robot inference time of 1.9s), to be of similar size to RT-1 (37M parameters for Gato vs. 35M for RT-1""  16M params for image tokenizer, 19M for the transformer",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2212.06817,CC BY 4.0 (Epoch AI)
model,CaLM,2022-12-19,University of Oxford,United Kingdom,Academia,Biology,parameters,86000000,86000000,Likely,"""We trained a large language model with 86M parameters""",,,,Fruit fly to honeybee,2.9e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2022.12.15.519894v1.full.pdf,CC BY 4.0 (Epoch AI)
model,Hybrid H3-2.7B,2022-12-28,"Stanford University,University at Buffalo",United States of America,Academia,Language,parameters,2700000000,2700000000,Likely,2.7B,,,,Honeybee to mouse,6.48e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2212.14052,CC BY 4.0 (Epoch AI)
model,VALL-E,2023-01-05,Microsoft,United States of America,Industry,"Audio,Speech",parameters,353000000,353000000,Speculative,"""Both the AR model and the NAR model have the same transformer architecture with 12 layers, 16 attention heads, an embedding dimension of 1024, a feed-forward layer dimension of 4096, and a dropout of 0.1""  Ben's script says that's 353M parameters, using n_block 12, d_model 1024, d_ff 4096, encoder ",,yes,,Fruit fly to honeybee,1.01e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2301.02111,CC BY 4.0 (Epoch AI)
model,DreamerV3,2023-01-10,"DeepMind,University of Toronto",Canada; United Kingdom,Mixed or other,Games,parameters,200000000,200000000,Likely,Table B1,,,,Fruit fly to honeybee,2.2032e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2301.04104v1,CC BY 4.0 (Epoch AI)
model,Nucleotide Transformer,2023-01-15,"Nvidia,Technical University of Munich,InstaDeep",Germany; United Kingdom; United States of America,Mixed or other,Biology,parameters,2500000000,2500000000,Likely,"""We built four distinct foundation language models of different sizes, ranging from 500M up to 2.5B parameters""",,,,Honeybee to mouse,8.08e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2023.01.11.523679v1.full.pdf,CC BY 4.0 (Epoch AI)
model,Ankh_large,2023-01-16,"Technical University of Munich,Columbia University",Germany; United States of America,Academia,Biology,parameters,1900000000,1900000000,Confident,"Figure 1 indicates 1.15B parameters, but both the huggingface model and a replication (https://huggingface.co/ElnaggarLab/ankh-large and https://www.biorxiv.org/content/10.1101/2023.07.05.547496v1.full.pdf) indicate 1.9B parameters. Notebook for counting params: https://colab.research.google.com/dri",,,,Honeybee to mouse,6.5e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2301.06568,CC BY 4.0 (Epoch AI)
model,MusicLM,2023-01-26,Google,United States of America,Industry,Audio,parameters,860000000,860000000,Confident,"""We use decoder-only Transformers for modeling the semantic stage and the acoustic stages of AudioLM. The models share the same architecture, composed of 24 layers, 16 attention heads, an embedding dimension of 1024, feed-forward layers of dimensionality 4096, dropout of 0.1, and relative positional",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2301.11325,CC BY 4.0 (Epoch AI)
model,DDPM-IP (CelebA),2023-01-27,Utrecht University,Netherlands,Academia,Image generation,parameters,295000000,295000000,Likely,"295M for CelebA model, per Table 9",,,,Fruit fly to honeybee,3.5e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2301.11706v3,CC BY 4.0 (Epoch AI)
model,BLIP-2 (Q-Former),2023-01-30,Salesforce Research,United States of America,Industry,"Vision,Language",parameters,1480000000,1480000000,Confident,"Q-Former has 188M params. The BLIP-2 system overall has ""54x fewer trainable parameters"" than Flamingo80B.",,,,Honeybee to mouse,1.20000000001e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2301.12597,CC BY 4.0 (Epoch AI)
model,Gen-1,2023-02-06,Runway,United States of America,Industry,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2302.03011,CC BY 4.0 (Epoch AI)
model,ProteinDT,2023-02-09,"University of California (UC) Berkeley,California Institute of Technology,University of Toronto,University of Wisconsin Madison,Texas A&M,Nvidia,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms)",Canada; United States of America,Mixed or other,"Biology,Language",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2302.04611,CC BY 4.0 (Epoch AI)
model,ViT-22B,2023-02-10,Google,United States of America,Industry,Vision,parameters,21743000000,21743000000,Confident,"21.743B, Table 1",,,,Honeybee to mouse,1.93248e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2302.05442v1,CC BY 4.0 (Epoch AI)
model,BASIC-L + Lion,2023-02-13,"Google,University of California Los Angeles (UCLA)",United States of America,Mixed or other,Vision,parameters,3070000000,3070000000,Confident,parameter count of original BASIC-L,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2302.06675v4,CC BY 4.0 (Epoch AI)
model,LLaMA-65B,2023-02-24,Meta AI,United States of America,Industry,Language,parameters,65200000000,65200000000,Confident,"Model card, table 1: https://github.com/facebookresearch/llama/blob/53011c3d7946dadb8274a4c5c7586ab54edf792d/MODEL_CARD.md",,,,Honeybee to mouse,5.5e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2302.13971,CC BY 4.0 (Epoch AI)
model,DiT-XL/2,2023-03-02,"New York University (NYU),University of California (UC) Berkeley",United States of America,Academia,Image generation,parameters,675000000,675000000,Confident,"675M  Table 4: ""Parameter and flop counts exclude the VAE model which contains 84M parameters across the encoder and decoder."" 675e6 not including VAE (likely frozen), 759e6 including VAE",,,,Honeybee to mouse,6e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2212.09748,CC BY 4.0 (Epoch AI)
model,AudioGen,2023-03-05,"Meta AI,Hebrew University of Jerusalem",Israel; United States of America,Mixed or other,Audio,parameters,1000000000,1000000000,Likely,"""We trained two sets of ALMs, one with 285M parameters (base) and the other with 1B parameters (large).""",,,,Honeybee to mouse,9.5e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2209.15352,CC BY 4.0 (Epoch AI)
model,PaLM-E,2023-03-06,"Google,TU Berlin",Germany; United States of America,Mixed or other,"Robotics,Vision,Language",parameters,562000000000,562000000000,Likely,562B,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.03378,CC BY 4.0 (Epoch AI)
model,Claude,2023-03-14,Anthropic,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/index/introducing-claude,CC BY 4.0 (Epoch AI)
model,Falcon-40B,2023-03-15,Technology Innovation Institute,United Arab Emirates,Mixed or other,Language,parameters,40000000000,40000000000,Confident,Model comes in 7B and 40B variants.,,,,Honeybee to mouse,2.4e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.16867; https://www.tii.ae/news/abu-dhabi-based-technology-innovation-institute-introduces-falcon-llm-foundational-large,CC BY 4.0 (Epoch AI)
model,GPT-4 (Mar 2023),2023-03-15,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,1800000000000,1800000000000,Likely,"Rumored to be 1.8T parameter MoE with 280B activated on the forward pass, per https://www.semianalysis.com/p/gpt-4-architecture-infrastructure. Other sources estimate 1.76T with 220B per forward pass https://web.archive.org/web/20230712123915/https://the-decoder.com/gpt-4-architecture-datasets-costs",yes,yes,,Mouse to cat,2.1e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.08774,CC BY 4.0 (Epoch AI)
model,LEP-AD,2023-03-15,"King Abdullah University of Science and Technology (KAUST),Karolinska Institute",Saudi Arabia; Sweden,Academia,Biology,parameters,3007381000,3007381000,Confident,"Uses ESM-2 3B. Table 2 gives details on the non-ESM layers. The GCN appears to have about 3.31M parameters and the linear layers should have 771k and 3.3M, respectively. So total is ~3.007B",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2023.03.14.532563v1.full.pdf,CC BY 4.0 (Epoch AI)
model,Gen-2,2023-03-20,Runway,United States of America,Industry,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.runwayml.com/gen2,CC BY 4.0 (Epoch AI)
model,PanGu-Σ,2023-03-20,Huawei Noah's Ark Lab,China,Industry,Language,parameters,1085000000000,1085000000000,Confident,"""In this work, we present PanGu-Σ , a large language model with sparse architecture containing 1.085 trillion parameters.""",yes,,,Mouse to cat,4.67e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.10845,CC BY 4.0 (Epoch AI)
model,Firefly,2023-03-21,Adobe,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://news.adobe.com/news/news-details/2023/Adobe-Unveils-Firefly-a-Family-of-new-Creative-Generative-AI/default.aspx,CC BY 4.0 (Epoch AI)
model,SigLIP 400M,2023-03-27,Google DeepMind,United States of America,Industry,Vision,parameters,400000000,400000000,Confident,Table 3,,,,Honeybee to mouse,4.9467301e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.15343,CC BY 4.0 (Epoch AI)
model,VideoMAE V2,2023-03-29,"Nanjing University,Shenzhen Institute of Advanced Technology,Shanghai AI Lab",China,Academia,Video,parameters,1000000000,1000000000,Confident,1B,,,,Honeybee to mouse,9.7e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.16727v2,CC BY 4.0 (Epoch AI)
model,BloombergGPT,2023-03-30,"Bloomberg,Johns Hopkins University",United States of America,Mixed or other,Language,parameters,50558868480,50558868480,Confident,,,,,Honeybee to mouse,2.36e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.17564,CC BY 4.0 (Epoch AI)
model,Segment Anything Model,2023-04-05,Meta AI,United States of America,Industry,Vision,parameters,636000000,636000000,Confident,"From Facebook website: https://segment-anything.com/ ""How big is the model? The image encoder has 632M parameters. The prompt encoder and mask decoder have 4M parameters.""",,,,Honeybee to mouse,7.8e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2304.02643,CC BY 4.0 (Epoch AI)
model,Incoder-6.7B,2023-04-09,"Facebook AI Research,University of Washington,University of California (UC) Berkeley,Carnegie Mellon University (CMU),Toyota Technological Institute at Chicago",France; United States of America,Mixed or other,Language,parameters,6700000000,6700000000,Confident,6.7B,,,,Honeybee to mouse,3.00001e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2204.05999,CC BY 4.0 (Epoch AI)
model,DINOv2,2023-04-14,"Facebook AI Research,INRIA",France; United States of America,Mixed or other,Vision,parameters,1140000000,1140000000,Confident,1.14B from https://huggingface.co/facebook/dinov2-giant,,,,Honeybee to mouse,7.41851136e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2304.07193,CC BY 4.0 (Epoch AI)
model,LLaVA,2023-04-17,"University of Wisconsin Madison,Microsoft Research,Columbia University",United States of America,Mixed or other,"Multimodal,Vision,Language",parameters,13000000000,13000000000,Confident,13B,,,,Honeybee to mouse,7.8049e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2304.08485,CC BY 4.0 (Epoch AI)
model,Agile Soccer Robot,2023-04-26,Google DeepMind,United States of America,Industry,Robotics,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2304.13653,CC BY 4.0 (Epoch AI)
model,ImageBind,2023-05-09,Meta AI,United States of America,Industry,"Multimodal,Vision,Audio,Language,Image generation,Speech",parameters,932000000,932000000,Likely,used ViT-Huge 630M as an image/video encoder and OpenCLIP-302m as text encoder,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2305.05665, https://github.com/facebookresearch/ImageBind",CC BY 4.0 (Epoch AI)
model,StarCoder,2023-05-09,"Hugging Face,ServiceNow,Northeastern University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),Carnegie Mellon University (CMU),Johns Hopkins University,Leipzig University,ScaDS.AI,Queen Mary University of London,Roblox,Sea AI Lab,Technion - Israel Institute of Technology,Monash University,CSIRO,Data61,McGill University,Saama,University of British Columbia (UBC),Massachusetts Institute of Technology (MIT),Technical University of Munich,IBM,University of Vermont,UnfoldML,SAP,University of Notre Dame,Columbia University,New York University (NYU),University of Allahabad,Discover Dollar,Toloka,Telefonica,Stanford University,Weizmann Institute of Science,Alan Turing Institute,Wellesley College,EleutherAI,Forschungszentrum Julich",Australia; Canada; Germany; India; Israel; Netherlands; Singapore; Spain; Sweden; United Kingdom; United States of America,Mixed or other,Language,parameters,15500000000,15500000000,Confident,"""We trained a 15.5B parameter model""",,,,Honeybee to mouse,8.46e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.06161,CC BY 4.0 (Epoch AI)
model,PaLM 2,2023-05-10,Google,United States of America,Industry,Language,parameters,340000000000,340000000000,Likely,"Model Architecture: ""PaLM-2 is a new state-of-the-art language model. We have small, medium, and large variants that use stacked layers based on the Transformer architecture, with varying parameters depending on model size. Further details of model size and architecture are withheld from external pu",,,,Mouse to cat,7.34e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.10403,CC BY 4.0 (Epoch AI)
model,InstructBLIP,2023-05-11,"Salesforce Research,Hong Kong University of Science and Technology (HKUST),Nanyang Technological University",Hong Kong; Singapore; United States of America,Mixed or other,"Multimodal,Language,Vision",parameters,13000000000,13000000000,Confident,13B form 2.6,,,,Honeybee to mouse,1.94e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.06500,CC BY 4.0 (Epoch AI)
model,Med-PaLM 2,2023-05-16,"Google Research,DeepMind",United Kingdom; United States of America,Industry,"Medicine,Language",parameters,340000000000,340000000000,Likely,from PaLM 2,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.09617,CC BY 4.0 (Epoch AI)
model,CoEdiT-xxl,2023-05-17,"University of Minnesota,Grammarly",United States of America,Mixed or other,Language,parameters,11000000000,11000000000,Likely,11B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2305.09857, https://huggingface.co/grammarly/coedit-large",CC BY 4.0 (Epoch AI)
model,ONE-PEACE,2023-05-18,"Alibaba,Huazhong University of Science and Technology",China,Mixed or other,"Multimodal,Vision,Speech,Language",parameters,4000000000,4000000000,Speculative,"""we propose ONE-PEACE, a model with 4B parameters""",,yes,,Honeybee to mouse,1.8e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.11172v1,CC BY 4.0 (Epoch AI)
model,CodeT5+,2023-05-20,Salesforce,United States of America,Industry,Language,parameters,16000000000,16000000000,Confident,"""We implemented a family of CodeT5+ models, with model sizes ranging from 220M to 16B""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.07922,CC BY 4.0 (Epoch AI)
model,Goat-7B,2023-05-23,National University of Singapore,Singapore,Academia,Language,parameters,7000000000,7000000000,Speculative,7B,,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.14201,CC BY 4.0 (Epoch AI)
model,DPO on Pythia-2.8B,2023-05-29,"Stanford University,CZ Biohub Network",United States of America,Academia,Language,parameters,2800000000,2800000000,Confident,same as base model,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.18290,CC BY 4.0 (Epoch AI)
model,PaLI-X,2023-05-29,Google Research,United States of America,Industry,"Multimodal,Language,Vision,Video",parameters,55000000000,55000000000,Likely,55B (table 1),,,,Honeybee to mouse,5.6e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.18565,CC BY 4.0 (Epoch AI)
model,LTM-1,2023-06-06,Magic,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://magic.dev/blog/ltm-1,CC BY 4.0 (Epoch AI)
model,MusicGen,2023-06-08,Meta AI,United States of America,Industry,Audio,parameters,3359000000,3359000000,Likely,"""We train autoregressive transformer models at different sizes: 300M, 1.5B, 3.3B parameters""  Uses EnCodec 32kHz (HF version has 59M params) for audio tokenization.",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2306.05284,CC BY 4.0 (Epoch AI)
model,GPT-3.5 Turbo,2023-06-13,OpenAI,United States of America,Industry,Language,parameters,20000000000,20000000000,Likely,20B parameters according to Table 1 in Microsoft's CODEFUSION paper: https://arxiv.org/pdf/2310.17680.pdf,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://platform.openai.com/docs/models/gpt-3.5-turbo,CC BY 4.0 (Epoch AI)
model,GPT-4 (Jun 2023),2023-06-13,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,1800000000000,1800000000000,Likely,"Rumored to be 1.8T parameter MoE with 280B activated on the forward pass, per https://www.semianalysis.com/p/gpt-4-architecture-infrastructure. Other sources estimate 1.76T with 220B per forward pass https://web.archive.org/web/20230712123915/https://the-decoder.com/gpt-4-architecture-datasets-costs",yes,yes,,Mouse to cat,2.1e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2303.08774,CC BY 4.0 (Epoch AI)
model,ERNIE 3.5,2023-06-27,Baidu,China,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,http://research.baidu.com/Blog/index-view?id=185,CC BY 4.0 (Epoch AI)
model,HyenaDNA,2023-06-27,"Stanford University,Harvard University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),University of Montreal / Université de Montréal",Canada; United States of America,Academia,Biology,parameters,6600000,6600000,Confident,"Table A.1 shows details, largest experiment is on far right.",,,,Below a fruit fly,1.811e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2306.15794,CC BY 4.0 (Epoch AI)
model,Stable Diffusion XL (SDXL),2023-07-04,Stability AI,United Kingdom,Industry,Image generation,parameters,3400000000,3400000000,Speculative,"""...result in a model size of 2.6B parameters in the UNet, see Tab. 1. The text encoders have a total size of 817M parameters.""",,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2307.01952,CC BY 4.0 (Epoch AI)
model,Pangu-Weather,2023-07-05,Huawei,China,Industry,Earth science,parameters,256000000,256000000,Confident,"4*64 million = 256M params  ""We trained four deep networks with lead times (the time difference between input and output) at 1 h, 3 h, 6 h and 24 h, respectively...   This modification increases the number of bias parameters by a factor of 527, with each 3D deep network containing approximately 64 m",,,,Fruit fly to honeybee,3.98e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.nature.com/articles/s41586-023-06185-3, https://www.huaweicloud.com/intl/en-us/news/20230707180809498.html,
https://www.huawei.com/en/news/2023/7/pangu-ai-model-nature-publish",CC BY 4.0 (Epoch AI)
model,InternLM,2023-07-06,"Shanghai AI Lab,SenseTime",China; Hong Kong,Mixed or other,Language,parameters,104000000000,104000000000,Confident,"""We present InternLM, a multilingual foundational language model with 104B parameters""",,,,Mouse to cat,9.984e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://github.com/InternLM/InternLM-techreport/blob/main/InternLM.pdf,CC BY 4.0 (Epoch AI)
model,xTrimoPGLM -100B,2023-07-06,"Tsinghua University,BioMap Research",China,Mixed or other,Biology,parameters,100000000000,100000000000,Confident,"Abstract: ""training xTrimoPGLM at an unprecedented scale of 100 billion parameters and 1 trillion training tokens""",,,,Mouse to cat,6.2e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2023.07.05.547496v4,CC BY 4.0 (Epoch AI)
model,Claude 2,2023-07-11,Anthropic,United States of America,Industry,Language,parameters,,,Speculative,,,,,,3.866e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.anthropic.com/index/claude-2, https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf",CC BY 4.0 (Epoch AI)
model,GPT3-2.7B (FlashAttention-2),2023-07-18,"Stanford University,Princeton University",United States of America,Academia,Language,parameters,2700000000,2700000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/pdf/2307.08691,CC BY 4.0 (Epoch AI)
model,Llama 2-70B,2023-07-18,Meta AI,United States of America,Industry,Language,parameters,70000000000,70000000000,Confident,"Llama has been released in 7B, 13B, 34B, and 70B variants.",,,,Honeybee to mouse,8.1e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/
https://arxiv.org/abs/2307.09288",CC BY 4.0 (Epoch AI)
model,Llama 2-7B,2023-07-18,Meta AI,United States of America,Industry,Language,parameters,7000000000,7000000000,Confident,"Llama has been released in 7B, 13B, and 70B variants.",,,,Honeybee to mouse,8.4e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/,CC BY 4.0 (Epoch AI)
model,AudioLM,2023-07-26,Google Research,United States of America,Industry,Audio,parameters,1500000000,1500000000,Speculative,"""We use identical decoder-only Transformers in all stages, with 12 layers, 16 attention heads, embedding dimension of 1024, feed-forward layer dimension of 4096 and dropout of 0.1, together with T5-style relative positional embeddings [38], resulting in a model parameter size of 0.3B per stage.""  Th",,yes,,Honeybee to mouse,3.9e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2209.03143,CC BY 4.0 (Epoch AI)
model,RT-2,2023-07-28,Google DeepMind,United States of America,Industry,Robotics,parameters,55000000000,55000000000,Confident,"""We train two specific instantiations of RT-2 that leverage pre-trained VLMs: (1) RT-2-PaLI-X is built from 5B and 55B PaLI-X (Chen et al., 2023a), and (2) RT-2-PaLM-E is built from 12B PaLM-E (Driess et al., 2023).""  55B and 12B have similar overall performance",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2307.15818,CC BY 4.0 (Epoch AI)
model,GGNN,2023-08-05,"Westlake University,Tsinghua University,Toyota Technological Institute at Chicago",China; United States of America,Academia,Biology,parameters,,,Confident,"ESM-2 650M is used as the main PLM, they run ablations with versions up to 3B. Unclear how many parameters are are in the geometric graph neural network module.",,,,,7.56e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s42003-023-05133-1,CC BY 4.0 (Epoch AI)
model,Qwen-VL,2023-08-24,Alibaba,China,Industry,"Multimodal,Language,Vision",parameters,9600000000,9600000000,Likely,9.6B total - Table 1,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2308.12966,CC BY 4.0 (Epoch AI)
model,PeptideBERT,2023-08-28,Carnegie Mellon University (CMU),United States of America,Academia,Biology,parameters,,,Confident,,,,,,4.9e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2309.03099,CC BY 4.0 (Epoch AI)
model,Jais,2023-08-29,"Cerebras Systems,Mohamed bin Zayed University of Artificial Intelligence (MBZUAI),Inception G42",United Arab Emirates; United States of America,Mixed or other,Language,parameters,13000000000,13000000000,Confident,"""With 13 billion parameters, they demonstrate better knowledge and reasoning capabilities in Arabic""",,,,Honeybee to mouse,4.8946763e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2308.16149,CC BY 4.0 (Epoch AI)
model,Swift,2023-08-30,Intel Labs,United States of America,Industry,Robotics,parameters,56804,56804,Likely,"The control network is an MLP with input dimension 31, two hidden layers of size 128, and an output of dimension 4. (31+1)*128+(128+1)*128+(128+1)*4 = 21124  Gate detector is a 6 layer U-net with  8*(3^3*3+1) + 16*(3^2*8+1) + 16*(3^2*16+1) + 16*(5^2*16+1) + 16*(7^2*16+1) + 16*(7^2*16+1) = 35680  356",,,,Below a fruit fly,5.337e+16,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-023-06419-4,CC BY 4.0 (Epoch AI)
model,Falcon-180B,2023-09-06,Technology Innovation Institute,United Arab Emirates,Mixed or other,Language,parameters,180000000000,180000000000,Confident,"""Falcon 180B is a super-powerful language model with 180 billion parameters""",,,,Mouse to cat,3.76e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://falconllm.tii.ae/falcon-180b.html; https://arxiv.org/abs/2311.16867,CC BY 4.0 (Epoch AI)
model,Robot Parkour,2023-09-12,"Shanghai Qi Zhi institute,Stanford University,Carnegie Mellon University (CMU),Tsinghua University",China; United States of America,Academia,Robotics,parameters,500000,500000,Confident,"Parkour policy details on page 8, table 11.",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2309.05665,CC BY 4.0 (Epoch AI)
model,AlphaMissense,2023-09-22,Google DeepMind,United States of America,Industry,Biology,parameters,93000000,93000000,Likely,"""The model architecture is similar to that of AlphaFold (21), with minor modifications"" Reference is to the AlphaFold 2 paper; that model had 93 million parameters",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.science.org/doi/10.1126/science.adg7492,CC BY 4.0 (Epoch AI)
model,GPT-4V,2023-09-25,OpenAI,United States of America,Industry,"Multimodal,Vision,Language",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/papers/GPTV_System_Card.pdf,CC BY 4.0 (Epoch AI)
model,Show-1,2023-09-27,National University of Singapore,Singapore,Academia,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2309.15818,CC BY 4.0 (Epoch AI)
model,Amazon Titan,2023-09-28,Amazon,United States of America,Industry,"Language,Image generation",parameters,200000000000,200000000000,Likely,200B dense model https://importai.substack.com/p/import-ai-365-wmd-benchmark-amazon,,,,Mouse to cat,4.8e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://aws.amazon.com/bedrock/titan/,CC BY 4.0 (Epoch AI)
model,GPT-3.5 Turbo Instruct,2023-09-28,OpenAI,United States of America,Industry,Language,parameters,20000000000,20000000000,Likely,20B parameters according to Table 1 in Microsoft's CODEFUSION paper: https://arxiv.org/pdf/2310.17680.pdf,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://platform.openai.com/docs/models/gpt-3.5-turbo,CC BY 4.0 (Epoch AI)
model,CTM (CIFAR-10),2023-10-01,"Stanford University,Sony",Japan; United States of America,Mixed or other,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.02279v1,CC BY 4.0 (Epoch AI)
model,FinGPT-13B,2023-10-07,"University of California Los Angeles (UCLA),Columbia University,New York University (NYU)",United States of America,Academia,Language,parameters,13000000000,13000000000,Likely,"Finetunes using LoRA, so only trains 3.67 million parameters",,,,Honeybee to mouse,1.6e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.04793; https://github.com/AI4Finance-Foundation/FinGPT,CC BY 4.0 (Epoch AI)
model,RoseTTAFold All-Atom (RFAA),2023-10-09,"University of Washington,Seoul National University,University of Sheffield",South Korea; United Kingdom; United States of America,Academia,Biology,parameters,,,Confident,,,,,,2.14e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.biorxiv.org/content/10.1101/2023.10.09.561603v1,CC BY 4.0 (Epoch AI)
model,Ferret (13B),2023-10-11,"Columbia University,Apple",United States of America,Mixed or other,"Multimodal,Language,Vision",parameters,13000000000,13000000000,Confident,13B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.07704,CC BY 4.0 (Epoch AI)
model,RT-2-X,2023-10-13,Google DeepMind,United States of America,Industry,Robotics,parameters,55000000000,55000000000,Confident,55B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.08864,CC BY 4.0 (Epoch AI)
model,ERNIE 4.0,2023-10-17,Baidu,China,Industry,"Multimodal,Language,Video,Image generation",parameters,,,Unknown,"""similar architecture with 3.5 version""  -interpreter dub at 01:25:08 https://www.youtube.com/watch?v=wYozcsavRuM",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.prnewswire.com/news-releases/baidu-launches-ernie-4-0-foundation-model-leading-a-new-wave-of-ai-native-applications-301958681.html,CC BY 4.0 (Epoch AI)
model,PaLI-3,2023-10-17,"Google DeepMind,Google Research,Google Cloud",United States of America,Industry,"Multimodal,Language,Vision",parameters,5000000000,5000000000,Confident,5B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.09199,CC BY 4.0 (Epoch AI)
model,DALL·E 3,2023-10-19,OpenAI,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/papers/dall-e-3.pdf,CC BY 4.0 (Epoch AI)
model,CODEFUSION (Python),2023-10-26,"Microsoft,Microsoft Research",United States of America,Industry,Language,parameters,75000000,75000000,Confident,Table 1,,,,Fruit fly to honeybee,7.92e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2310.17680 (was withdrawn)

https://www.microsoft.com/en-us/research/wp-content/uploads/2023/11/CodeFusion-Revised-CameraReady.pdf",CC BY 4.0 (Epoch AI)
model,DiT-XL/2 + CADS,2023-10-26,"ETH Zurich,Disney Research",Switzerland; United States of America,Mixed or other,Image generation,parameters,675000000,675000000,Likely,original parameter count for DiT-XL/2,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.17347v2,CC BY 4.0 (Epoch AI)
model,ChatGLM3-6B,2023-10-27,Z.ai (Zhipu AI),China,Industry,"Multimodal,Language,Vision",parameters,6000000000,6000000000,Speculative,6B from https://arxiv.org/abs/2406.12793,,yes,,Honeybee to mouse,5.04e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.zhipuai.cn/en/news/76

https://huggingface.co/zai-org/chatglm3-6b",CC BY 4.0 (Epoch AI)
model,Skywork-13B,2023-10-30,Kunlun Inc.,China,Industry,Language,parameters,13000000000,13000000000,Confident,13B,,,,Honeybee to mouse,2.5e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2310.19341,CC BY 4.0 (Epoch AI)
model,Cohere Embed,2023-11-02,Cohere,Canada,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://txt.cohere.com/introducing-embed-v3/,CC BY 4.0 (Epoch AI)
model,Yi-34B,2023-11-02,01.AI,China,Industry,Language,parameters,34000000000,34000000000,Confident,34b,,,,Honeybee to mouse,6.1e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2403.04652,CC BY 4.0 (Epoch AI)
model,BLUUMI,2023-11-03,"University of Turku,Hugging Face",Finland; United States of America,Mixed or other,Language,parameters,176000000000,176000000000,Likely,176 billion,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.05640,CC BY 4.0 (Epoch AI)
model,RT-Trajectory,2023-11-03,"Google DeepMind,University of California San Diego,Stanford University",United States of America,Mixed or other,Robotics,parameters,,,Unknown,seems to be based on the RT-1 architecture (35M parameters) with some modifications (section 3.3),,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.01977,CC BY 4.0 (Epoch AI)
model,Grok-1,2023-11-04,xAI,United States of America,Industry,Language,parameters,314000000000,314000000000,Likely,"""314B parameter Mixture-of-Experts model with 25% of the weights active on a given token"". So effectively 78B parameters  Mixture of 8 experts: https://github.com/xai-org/grok-1",yes,,,Mouse to cat,2.90000000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://x.ai/model-card/, https://x.ai/blog/grok-os",CC BY 4.0 (Epoch AI)
model,LLaVA 1.5,2023-11-05,"University of Wisconsin Madison,Microsoft Research",United States of America,Mixed or other,"Multimodal,Language,Vision",parameters,13000000000,13000000000,Confident,"from abstract ""Our final 13B checkpoint uses merely 1.2M publicly available data, and finishes full training in ~1 day on a single 8-A100 node. """,,,,Honeybee to mouse,7.807e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2310.03744,
",CC BY 4.0 (Epoch AI)
model,CogVLM-17B,2023-11-06,"Tsinghua University,Z.ai (Zhipu AI),Beihang University",China,Mixed or other,"Multimodal,Vision,Language",parameters,17000000000,17000000000,Confident,"CogVLM-17B has 10 billion vision parameters and 7 billion language parameters. However, ""the total number of trainable parameters is 6.5B"".  ""CogVLM model comprises four fundamental components: a vision transformer (ViT) encoder, an MLP adapter, a pretrained large language model (GPT), and a visual ",,,,Honeybee to mouse,6.331e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2311.03079
https://huggingface.co/THUDM/cogvlm-chat-hf
https://github.com/THUDM/CogVLM
",CC BY 4.0 (Epoch AI)
model,GPT-4 Turbo (Nov 2023),2023-11-06,OpenAI,United States of America,Industry,"Multimodal,Vision,Language,Image generation",parameters,,,Unknown,Not known. Maybe smaller/sparser than GPT-4.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/blog/new-models-and-developer-products-announced-at-devday,CC BY 4.0 (Epoch AI)
model,OmniVec,2023-11-07,TensorTour,United States of America,Industry,"Multimodal,Vision,Speech,Language,Video,3D modeling",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.05709v1,CC BY 4.0 (Epoch AI)
model,mPLUG-Owl2,2023-11-07,Alibaba,China,Industry,"Vision,Language,Multimodal",parameters,7120000000,7120000000,Speculative,"""As depicted in Figure 2, our model, referred to as mPLUGOwl2, is composed of three main components: a fundamental vision encoder, a visual abstractor, and a language decoder. Specifically, we utilize ViT-L/14 as the vision encoder and LLaMA-2-7B [58] as the language decoder"" ViT-L/14 has 123M param",,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.04257,CC BY 4.0 (Epoch AI)
model,MultiBand Diffusion,2023-11-08,"Meta AI,Hebrew University of Jerusalem,LORIA",France; Israel; United States of America,Mixed or other,"Audio,Speech",parameters,,,Confident,,,,,,2.6e+19,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2308.02560,CC BY 4.0 (Epoch AI)
model,RoFormer,2023-11-08,Zhuiyi Technology,China,Industry,Language,parameters,110000000,110000000,Confident,"""we use bert-base""  BERT base has 110M parameters from https://arxiv.org/pdf/1810.04805",,,,Fruit fly to honeybee,2.162688e+18,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2104.09864,CC BY 4.0 (Epoch AI)
model,SPHINX (Llama 2 13B),2023-11-13,"Shanghai AI Lab,Chinese University of Hong Kong (CUHK),ShanghaiTech University",China; Hong Kong,Academia,"Vision,Language,Multimodal",parameters,19900000000,19900000000,Likely,"SPHINX + Llama 2 13B SPHINX component involves four vision encoders: - CLIP - ViT - CLIP - ConvNeXt V2 (89M to 659M params, depending on size) - DinoV2 - ViT (22M to 1.14B params, depending on size) - Q-former (188M params) Also involves to projection networks  Huggingface Hub model files appear to ",,,,Honeybee to mouse,3.04e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.07575,CC BY 4.0 (Epoch AI)
model,Volcano 13B,2023-11-13,"Korea University,Korea Advanced Institute of Science and Technology (KAIST),LG",South Korea,Mixed or other,"Language,Multimodal,Vision",parameters,13000000000,13000000000,Likely,13B,,,,Honeybee to mouse,4.56e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.07362,CC BY 4.0 (Epoch AI)
model,GraphCast,2023-11-14,Google DeepMind,United States of America,Industry,Earth science,parameters,,,Speculative,Not mentioned in paper.,,,,,2.1000000000000002e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.science.org/doi/epdf/10.1126/science.adi2336

https://arxiv.org/abs/2212.12794",CC BY 4.0 (Epoch AI)
model,Qwen-Audio-Chat,2023-11-14,Alibaba,China,Industry,"Language,Speech,Audio",parameters,8460000000,8460000000,Likely,"the model has two components - audio and language. 670M + 7.7B = 8.46B ""The audio encoder is composed of 640M parameters""  ""Qwen-Audio incorporates a large language model as its foundational component. The model is initialized using pre-trained weights derived from Qwen-7B (Bai et al., 2023a). Qwen-",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2311.07919,CC BY 4.0 (Epoch AI)
model,Nemotron-3-8B,2023-11-15,Nvidia,United States of America,Industry,Language,parameters,8000000000,8000000000,Confident,,,,,Honeybee to mouse,1.8e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://developer.nvidia.com/blog/nvidia-ai-foundation-models-build-custom-enterprise-chatbots-and-co-pilots-with-production-ready-llms/

https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/nemotron-3-8b-base-4k",CC BY 4.0 (Epoch AI)
model,AndesGPT,2023-11-16,Oppo Mobile Telecommunications,China,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.oppo.com/en/newsroom/press/2023-oppo-developers-conference-odc23/,CC BY 4.0 (Epoch AI)
model,Claude 2.1,2023-11-21,Anthropic,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/index/claude-2-1,CC BY 4.0 (Epoch AI)
model,Inflection-2,2023-11-22,Inflection AI,United States of America,Industry,Language,parameters,,,Confident,,,,,,1.001e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://inflection.ai/inflection-2,CC BY 4.0 (Epoch AI)
model,GNoME for crystal discovery,2023-11-29,Google DeepMind,United States of America,Industry,Materials science,parameters,16240000,16240000,Likely,"""The pretrained potential has 16.24 million parameters."" This refers to the GNoME network, which is a ""Gaussian Network Model of Energy"" for predicting crystal potential energy of new crystals.",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-023-06735-9,CC BY 4.0 (Epoch AI)
model,PPLX-70B-Online,2023-11-29,Perplexity,United States of America,Industry,Language,parameters,70000000000,70000000000,Likely,70B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.perplexity.ai/blog/introducing-pplx-online-llms,CC BY 4.0 (Epoch AI)
model,Qwen-72B,2023-11-30,Alibaba,China,Industry,Language,parameters,72000000000,72000000000,Confident,72B,,,,Honeybee to mouse,1.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/Qwen/Qwen-72B,CC BY 4.0 (Epoch AI)
model,Mamba-24M (SC09),2023-12-01,"Carnegie Mellon University (CMU),Princeton University",United States of America,Academia,Speech,parameters,23400000,23400000,Confident,Table 4,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.00752,CC BY 4.0 (Epoch AI)
model,Gemini 1.0 Pro,2023-12-06,Google DeepMind,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Speculative,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf,CC BY 4.0 (Epoch AI)
model,Gemini 1.0 Ultra,2023-12-06,Google DeepMind,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Speculative,,,,,,5.0000000001e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf,CC BY 4.0 (Epoch AI)
model,Llama Guard,2023-12-07,Meta AI,United States of America,Industry,Language,parameters,7000000000,7000000000,Confident,7B,,,,Honeybee to mouse,1.6e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.06674,CC BY 4.0 (Epoch AI)
model,SeamlessM4T,2023-12-08,"Facebook,INRIA,University of California (UC) Berkeley",France; United States of America,Mixed or other,"Speech,Language",parameters,2300000000,2300000000,Confident,2.3B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2312.05187, https://huggingface.co/facebook/seamless-m4t-v2-large",CC BY 4.0 (Epoch AI)
model,Mixtral 8x7B,2023-12-11,Mistral AI,France,Industry,Language,parameters,46700000000,46700000000,Speculative,"46.7B *sparse* params. 12.9B params used on average:  ""Concretely, Mixtral has 46.7B total parameters but only uses 12.9B parameters per token. It, therefore, processes input and generates output at the same speed and for the same cost as a 12.9B model.""",yes,yes,,Honeybee to mouse,7.74e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://mistral.ai/news/mixtral-of-experts/, https://arxiv.org/abs/2401.04088",CC BY 4.0 (Epoch AI)
model,W.A.L.T,2023-12-11,"Stanford University,Google Research,Georgia Institute of Technology",United States of America,Mixed or other,Video,parameters,4719000000,4719000000,Confident,"""We train our base model at resolution 17 × 128 × 128 (3B parameters), and two 2× cascaded super-resolution models for 17 × 128 × 224 → 17 × 256 × 448 (L, 1.3B, p = 2) and 17 × 256 × 448 → 17 × 512 × 896 (L, 419M, p = 2) respectively""  3B + 1.3B + 419M = 4.719B",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.06662,CC BY 4.0 (Epoch AI)
model,VILA-13B,2023-12-12,"Nvidia,Massachusetts Institute of Technology (MIT)",United States of America,Mixed or other,"Multimodal,Language,Vision",parameters,13350839296,13350839296,Confident,https://huggingface.co/Efficient-Large-Model/VILA-13b/tree/main?show_file_info=model.safetensors.index.json ,,,,Honeybee to mouse,2.3003136e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2312.07533
https://huggingface.co/Efficient-Large-Model/VILA-13b",CC BY 4.0 (Epoch AI)
model,CogAgent,2023-12-14,"Tsinghua University,Z.ai (Zhipu AI)",China,Mixed or other,"Vision,Language",parameters,18000000000,18000000000,Likely,,,,,Honeybee to mouse,6.707e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.08914,CC BY 4.0 (Epoch AI)
model,FunSearch,2023-12-14,Google DeepMind,United States of America,Industry,"Language,Search",parameters,15000000000,15000000000,Speculative,"From the section called ""Pretrained LLM"": ""We use Codey, an LLM built on top of the PaLM2 model family... Because FunSearch relies on sampling from an LLM extensively, an important performance-defining tradeoff is between the quality of the samples and the inference speed of the LLM. In practice, we",,yes,,Honeybee to mouse,3.87e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.nature.com/articles/s41586-023-06924-6
https://deepmind.google/discover/blog/funsearch-making-new-discoveries-in-mathematical-sciences-using-large-language-models/",CC BY 4.0 (Epoch AI)
model,Gemini Nano-1,2023-12-19,Google DeepMind,United States of America,Industry,"Multimodal,Language,Vision,Audio",parameters,1800000000,1800000000,Confident,1.8B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.11805,CC BY 4.0 (Epoch AI)
model,Gemini Nano-2,2023-12-19,Google DeepMind,United States of America,Industry,"Multimodal,Language,Vision,Audio",parameters,3250000000,3250000000,Confident,3.25B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.11805,CC BY 4.0 (Epoch AI)
model,nekomata-14b,2023-12-21,rinna,Japan,Industry,Language,parameters,14200000000,14200000000,Confident,Source: https://huggingface.co/rinna/nekomata-14b,,,,Honeybee to mouse,2.5562e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2404.01657,CC BY 4.0 (Epoch AI)
model,GQA-8-XXL,2023-12-23,Google Research,United States of America,Industry,Language,parameters,11000000000,11000000000,Confident,same as base model,,,,Honeybee to mouse,3.4912896e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2305.13245,CC BY 4.0 (Epoch AI)
model,CoRe,2023-12-29,Tsinghua University,China,Academia,"Mathematics,Language",parameters,12400000000,12400000000,Speculative,"""Since the default setting consists of two GPT-J (6B) and a DeBERTa-large (0.4B), we note our backbone as “GPT-J 12B”, which implies around 12.4 billion parameters in total. """,,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2210.16257,CC BY 4.0 (Epoch AI)
model,Kimi Explorer,2024-01-01,Moonshot,China,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.53ai.com/news/LargeLanguageModel/2024101137012.html,CC BY 4.0 (Epoch AI)
model,Palmyra X 003,2024-01-01,Writer,United States of America,Industry,Language,parameters,72000000000,72000000000,,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://writer.com/llms/palmyra-x-003/,CC BY 4.0 (Epoch AI)
model,AlphaGeometry,2024-01-17,"Google DeepMind,New York University (NYU)",United States of America,Mixed or other,Mathematics,parameters,151000000,151000000,Confident,"""Overall, the transformer has 151 million parameters, excluding embedding layers at its input and output heads.""",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-023-06747-5,CC BY 4.0 (Epoch AI)
model,Qwen-VL-Max,2024-01-25,Alibaba,China,Industry,"Multimodal,Language,Vision",parameters,7000000000,7000000000,Confident,"Not stated. Qwen-VL (less capable, presumably smaller version) is 9.6B  Upd: 7B parameters mentioned here https://github.com/QwenLM/Qwen-VL#qwen-vl-plus",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen-vl/,CC BY 4.0 (Epoch AI)
model,Qwen1.5-72B,2024-02-04,Alibaba,China,Industry,Language,parameters,72000000000,72000000000,Confident,72B,,,,Honeybee to mouse,1.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen1.5/,CC BY 4.0 (Epoch AI)
model,Aya,2024-02-12,"Cohere Labs (formerly Cohere for AI),Brown University,Cohere,Carnegie Mellon University (CMU),Massachusetts Institute of Technology (MIT)",Canada; United States of America,Mixed or other,Language,parameters,13000000000,13000000000,Speculative,13B  - fine tune of mT5 - from last page - model card ,,yes,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2402.07827,CC BY 4.0 (Epoch AI)
model,Gemini 1.5 Pro,2024-02-15,Google DeepMind,United States of America,Industry,"Language,Multimodal",parameters,,,Speculative,MoE architecture,yes,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf,CC BY 4.0 (Epoch AI)
model,Sora,2024-02-15,OpenAI,United States of America,Industry,"Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/video-generation-models-as-world-simulators/,CC BY 4.0 (Epoch AI)
model,Stable Diffusion 3,2024-02-22,Stability AI,United Kingdom,Industry,Image generation,parameters,8000000000,8000000000,Confident,,,,,Honeybee to mouse,5.0000000000000004e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2403.03206
https://stability.ai/news/stable-diffusion-3",CC BY 4.0 (Epoch AI)
model,MegaScale (Production),2024-02-23,"ByteDance,Peking University",China,Mixed or other,Language,parameters,530000000000,530000000000,Speculative,"Production run is stated to have ""hundreds of billions of parameters"". Since the authors also do a number of experiments with a 530B model, I speculate they've used 530B for the production model.",,yes,,Mouse to cat,3.9e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2402.15627,CC BY 4.0 (Epoch AI)
model,Mistral Large,2024-02-26,Mistral AI,France,Industry,Language,parameters,,,Speculative,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mistral.ai/news/mistral-large/,CC BY 4.0 (Epoch AI)
model,Aramco Metabrain AI,2024-03-04,Saudi Aramco,Saudi Arabia,Mixed or other,Language,parameters,250000000000,250000000000,Likely,"""It has 250 billion parameters that are adjustable during training to generate outputs or make predictions.""",,,,Mouse to cat,1.05e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.offshore-technology.com/news/saudi-aramco-unveils-industry-first-generative-ai-model/,CC BY 4.0 (Epoch AI)
model,Claude 3 Opus,2024-03-04,Anthropic,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Speculative,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf,CC BY 4.0 (Epoch AI)
model,Claude 3 Sonnet,2024-03-04,Anthropic,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf,CC BY 4.0 (Epoch AI)
model,Inflection-2.5,2024-03-07,Inflection AI,United States of America,Industry,Language,parameters,,,Likely,,,,,,8.000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://inflection.ai/inflection-2-5,CC BY 4.0 (Epoch AI)
model,ManiGaussian,2024-03-13,"Tsinghua University,Nanyang Technological University,Carnegie Mellon University (CMU)",China; Singapore; United States of America,Academia,"Robotics,Vision,Video",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2403.08321
https://guanxinglu.github.io/ManiGaussian/",CC BY 4.0 (Epoch AI)
model,MM1-30B,2024-03-14,Apple,United States of America,Industry,"Multimodal,Language,Vision",parameters,30000000000,30000000000,Likely,30B,,,,Honeybee to mouse,4.86e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2403.09611,CC BY 4.0 (Epoch AI)
model,DBRX,2024-03-27,Databricks,United States of America,Industry,Language,parameters,132000000000,132000000000,Confident,132B mixture of experts. 36B parameters active per inference,yes,,,Mouse to cat,2.6e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm,CC BY 4.0 (Epoch AI)
model,ReALM,2024-03-29,Apple,United States of America,Industry,Language,parameters,3000000000,3000000000,Confident,Fine-tuned FLAN-T5 models ranging from 80M to 3B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2403.20329,CC BY 4.0 (Epoch AI)
model,GPT-4 Turbo (Apr 2024),2024-04-09,OpenAI,United States of America,Industry,"Multimodal,Vision,Language,Image generation",parameters,,,Unknown,Not known. Maybe smaller/sparser than GPT-4.,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.com/OpenAI/status/1778574613813006610,CC BY 4.0 (Epoch AI)
model,Reka Core,2024-04-15,Reka AI,United States of America,Industry,"Multimodal,Language,Vision,Video,Speech",parameters,67000000000,67000000000,Speculative,,,yes,,Honeybee to mouse,8.400010000000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2404.12387,CC BY 4.0 (Epoch AI)
model,Llama 3-70B,2024-04-18,Meta AI,United States of America,Industry,Language,parameters,70000000000,70000000000,Confident,,,,,Honeybee to mouse,7.861e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/blog/meta-llama-3/,CC BY 4.0 (Epoch AI)
model,GenCast,2024-05-01,Google DeepMind,United States of America,Industry,Earth science,parameters,,,Confident,,,,,,8.169984e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2312.15796,CC BY 4.0 (Epoch AI)
model,VILA1.5-13B,2024-05-03,"Nvidia,Massachusetts Institute of Technology (MIT)",United States of America,Mixed or other,"Multimodal,Language,Vision,Video",parameters,13493916736,13493916736,Confident,https://huggingface.co/Efficient-Large-Model/VILA1.5-13b/blob/main/llm/model.safetensors.index.json https://huggingface.co/Efficient-Large-Model/VILA1.5-13b/tree/main/vision_tower https://huggingface.co/Efficient-Large-Model/VILA1.5-13b/blob/main/mm_projector/model.safetensors  llm: 13015864320 visi,,,,Honeybee to mouse,2.3003136e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/Efficient-Large-Model/VILA1.5-13b
https://github.com/NVlabs/VILA/tree/bbc609baf326b1b49b93450b48edc516db3737fc/scripts/v1_5/release/13b
https://developer.nvidia.com/blog/visual-language-models-on-nvidia-hardware-with-vila/
",CC BY 4.0 (Epoch AI)
model,AlphaFold 3,2024-05-08,"Google DeepMind,Isomorphic Labs",United Kingdom; United States of America,Industry,Biology,parameters,,,Confident,,,,,,4.1405645e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.nature.com/articles/s41586-024-07487-w,CC BY 4.0 (Epoch AI)
model,GPT-4o,2024-05-13,OpenAI,United States of America,Industry,"Multimodal,Language,Audio,Speech,Vision",parameters,,,Speculative,Not known.  Inference costs in the API are 2x cheaper than GPT-4 Turbo,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://openai.com/index/hello-gpt-4o/ 
https://openai.com/index/gpt-4o-system-card/",CC BY 4.0 (Epoch AI)
model,Yi-Large,2024-05-13,01.AI,China,Industry,Language,parameters,100000000000,100000000000,Speculative,"""Yi-Large is a software over-the-air-driven closed-source large model with a parameter of over 100 billion tokens."" from https://www.chinadaily.com.cn/a/202405/13/WS6641abd1a31082fc043c6ccd.html",,yes,,Mouse to cat,1.8e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,,CC BY 4.0 (Epoch AI)
model,GLM-4 (0520),2024-05-20,Z.ai (Zhipu AI),China,Industry,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2406.12793,CC BY 4.0 (Epoch AI)
model,Octo-Base,2024-05-20,"University of California (UC) Berkeley,Stanford University,Carnegie Mellon University (CMU),DeepMind",United Kingdom; United States of America,Mixed or other,Robotics,parameters,93000000,93000000,Confident,Source: https://arxiv.org/abs/2405.12213 ,,,,Fruit fly to honeybee,5.85e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2405.12213 ,CC BY 4.0 (Epoch AI)
model,ALLaM adapted 70B,2024-05-21,Saudi Data and Artificial Intelligence Authority,Saudi Arabia,Mixed or other,Language,parameters,70000000000,70000000000,Confident,,,,,Honeybee to mouse,1.062e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2407.15390,CC BY 4.0 (Epoch AI)
model,Qwen2-72B,2024-06-07,Alibaba,China,Industry,Language,parameters,72710000000,72710000000,Confident,"72.71B parameters in total, of which 70.21B are non-embedding parameters",,,,Honeybee to mouse,3.02e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://qwenlm.github.io/blog/qwen2/ 
https://arxiv.org/abs/2407.10671 ",CC BY 4.0 (Epoch AI)
model,OpenVLA,2024-06-13,"Stanford University,University of California (UC) Berkeley,Toyota Research Institute,Google DeepMind,Massachusetts Institute of Technology (MIT),Physical Intelligence",United States of America,Mixed or other,"Robotics,Vision,Language",parameters,7188100000,7188100000,Confident,"Based on a Prismatic-7B VLM backbone, which itself is comprised of 600M parameter vision encoder (DinoV2 + SigLIP) plus Llama-2 7B. Table 1 indicates 7.1881 billion trainable parameters",,,,Honeybee to mouse,1.1e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://openvla.github.io/
https://arxiv.org/abs/2406.09246",CC BY 4.0 (Epoch AI)
model,Nemotron-4 340B,2024-06-14,Nvidia,United States of America,Industry,Language,parameters,340000000000,340000000000,Confident,340B,,,,Mouse to cat,1.8e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blogs.nvidia.com/blog/nemotron-4-synthetic-data-generation-llm-training/ ,CC BY 4.0 (Epoch AI)
model,DeepSeek-Coder-V2 236B,2024-06-17,DeepSeek,China,Industry,Language,parameters,236000000000,236000000000,Confident,Mixture of experts model. 21B parameters activated per token.,yes,,,Mouse to cat,1.2852e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://github.com/deepseek-ai/DeepSeek-Coder-V2,CC BY 4.0 (Epoch AI)
model,Claude 3.5 Sonnet,2024-06-20,Anthropic,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Speculative,,,,,,2.700000000000001e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf,CC BY 4.0 (Epoch AI)
model,Cambrian-1-34B,2024-06-24,New York University (NYU),United States of America,Academia,"Multimodal,Vision,Language",parameters,34000000000,34000000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2406.16860,CC BY 4.0 (Epoch AI)
model,ESM3 (98B),2024-06-25,"EvolutionaryScale,University of California (UC) Berkeley",United States of America,Mixed or other,Biology,parameters,98500000000,98500000000,Confident,98.5 billion (Table S1),,,,Mouse to cat,1.07e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.evolutionaryscale.ai/blog/esm3-release ,CC BY 4.0 (Epoch AI)
model,Ernie 4.0 Turbo,2024-06-28,Baidu,China,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.reuters.com/technology/artificial-intelligence/baidu-launches-upgraded-ai-model-says-user-base-hits-300-mln-2024-06-28/,CC BY 4.0 (Epoch AI)
model,SenseChat 5.5,2024-07-06,SenseTime,Hong Kong,Industry,"Multimodal,Language,Vision",parameters,600000000000,600000000000,Confident,https://en.tmtpost.com/post/7159076,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.sensetime.com/en/news-detail/51168278?categoryId=1072,CC BY 4.0 (Epoch AI)
model,DeepL LLM,2024-07-16,DeepL,Germany,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.deepl.com/en/blog/next-gen-language-model,CC BY 4.0 (Epoch AI)
model,Mathstral,2024-07-16,Mistral AI,France,Industry,"Mathematics,Language",parameters,7000000000,7000000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/mistralai/Mathstral-7B-v0.1
https://mistral.ai/news/mathstral",CC BY 4.0 (Epoch AI)
model,GPT-4o mini,2024-07-18,OpenAI,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Speculative,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/,CC BY 4.0 (Epoch AI)
model,Llama 3.1-405B,2024-07-23,Meta AI,United States of America,Industry,Language,parameters,405000000000,405000000000,Confident,405B,,,,Mouse to cat,3.8e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/research/publications/the-llama-3-herd-of-models/,CC BY 4.0 (Epoch AI)
model,Mistral Large 2,2024-07-24,Mistral AI,France,Industry,Language,parameters,123000000000,123000000000,Likely,,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mistral.ai/news/mistral-large-2407/,CC BY 4.0 (Epoch AI)
model,AFM-on-device,2024-07-29,Apple,United States of America,Industry,Language,parameters,2730000000,2730000000,Confident,"Table 1, sum of non-embedding and embedding parameters",,,,Honeybee to mouse,4.5126e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://machinelearning.apple.com/research/apple-intelligence-foundation-language-models,CC BY 4.0 (Epoch AI)
model,AFM-server,2024-07-29,Apple,United States of America,Industry,Language,parameters,,,Likely,,,,,,4.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://machinelearning.apple.com/research/apple-intelligence-foundation-language-models,CC BY 4.0 (Epoch AI)
model,GPT-4o (Aug 2024),2024-08-06,OpenAI,United States of America,Industry,"Multimodal,Language,Audio,Speech,Vision",parameters,,,Speculative,Not known.  Inference costs in the API are 2x cheaper than GPT-4 Turbo,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://help.openai.com/en/articles/9624314-model-release-notes,CC BY 4.0 (Epoch AI)
model,LLaVA-OV-72B,2024-08-06,"ByteDance,Nanyang Technological University,Chinese University of Hong Kong (CUHK),Hong Kong University of Science and Technology (HKUST)",China; Hong Kong; Singapore,Mixed or other,"Multimodal,Vision,Language,Video",parameters,72000000000,72000000000,Confident,,,,,Honeybee to mouse,3.036551985824e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2408.03326,CC BY 4.0 (Epoch AI)
model,Table Tennis Agent,2024-08-07,Google DeepMind,United States of America,Industry,Robotics,parameters,185000,185000,Likely,"17 low level controllers with 10k parameters each:  ""Each policy is a dilated-gated CNN [22] following the architecture in [23] with 10k parameters... The final system contained 17 LLCs""  One high-level controller with 4.5k parameters: ""The style policy architecture, similar to the LLC but with only",,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://deepmind.google/research/publications/107741/
https://arxiv.org/abs/2408.03906",CC BY 4.0 (Epoch AI)
model,Grok-2,2024-08-13,xAI,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Confident,,,,,,2.96e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/blog/grok-2,CC BY 4.0 (Epoch AI)
model,Jamba 1.5-Large,2024-08-22,AI21 Labs,Israel,Industry,Language,parameters,398000000000,398000000000,Confident,94B active/398B total,yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2408.12570
https://www.ai21.com/blog/announcing-jamba-model-family
https://huggingface.co/ai21labs/AI21-Jamba-1.5-Large",CC BY 4.0 (Epoch AI)
model,GLM-4-Plus,2024-08-29,Z.ai (Zhipu AI),China,Industry,Language,parameters,,,Speculative,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://bigmodel.cn/dev/howuse/glm-4,CC BY 4.0 (Epoch AI)
model,Hairuo,2024-08-29,Inspur,China,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://encloud.inspur.com/hairuo/cp/index.html

https://www.eguizhou.gov.cn/guiyang/2024-08/29/c_1016827.htm",CC BY 4.0 (Epoch AI)
model,AlphaProteo,2024-09-05,Google DeepMind,United States of America,Industry,Biology,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaproteo-generates-novel-proteins-for-biology-and-health-research/AlphaProteo2024.pdf,CC BY 4.0 (Epoch AI)
model,Harrison.rad.1,2024-09-05,Harrison.ai,Australia,Industry,"Vision,Medicine,Language,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://harrison.ai/news/harrison-ai-launches-world-leading-ai-model-to-transform-healthcare/,CC BY 4.0 (Epoch AI)
model,Hunyuan Turbo,2024-09-05,Tencent,China,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://zhidx.com/p/442250.html,CC BY 4.0 (Epoch AI)
model,DeepSeek-V2.5,2024-09-06,DeepSeek,China,Industry,Language,parameters,236000000000,236000000000,Confident,"21B active params, 236B total",yes,,,Mouse to cat,1.7892e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/deepseek-ai/DeepSeek-V2.5,CC BY 4.0 (Epoch AI)
model,o1-mini,2024-09-12,OpenAI,United States of America,Industry,Language,parameters,,,Unknown,"Can't get an exact estimate, but we suspect total parameter count around 60B-120B, active parameters around 10B-30B.   Given these models are served at 150-200 tok/s, at $4.40/Mtok output, inference economics (https://epoch.ai/blog/inference-economics-of-language-models) suggests total parameter cou",yes,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/learning-to-reason-with-llms/,CC BY 4.0 (Epoch AI)
model,o1-preview,2024-09-12,OpenAI,United States of America,Industry,"Language,Mathematics,Biology",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-openai-o1-preview/,CC BY 4.0 (Epoch AI)
model,Qwen2.5-32B,2024-09-17,Alibaba,China,Industry,Language,parameters,32500000000,32500000000,Confident,32.5B,,,,Honeybee to mouse,3.51e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen2.5/ ,CC BY 4.0 (Epoch AI)
model,Oryx 34B,2024-09-19,"Tsinghua University,Tencent,Nanyang Technological University",China; Singapore,Mixed or other,"Multimodal,Vision,3D modeling,Video,Language",parameters,34000000000,34000000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2409.12961v1,CC BY 4.0 (Epoch AI)
model,Qwen2.5 Instruct (72B),2024-09-19,Alibaba,China,Industry,Language,parameters,72700000000,72700000000,Confident,Number of Parameters: 72.7B Number of Paramaters (Non-Embedding): 70.0B,,,,Honeybee to mouse,7.8516e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen2.5/,CC BY 4.0 (Epoch AI)
model,Qwen2.5-72B,2024-09-19,Alibaba,China,Industry,Language,parameters,72700000000,72700000000,Confident,72.7B,,,,Honeybee to mouse,7.8e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen2.5/,CC BY 4.0 (Epoch AI)
model,Telechat2-115B,2024-09-20,China Telecom,China,Industry,Language,parameters,115000000000,115000000000,Confident,,,,,Mouse to cat,6.9e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/Tele-AI/TeleChat2-115B,CC BY 4.0 (Epoch AI)
model,Llama 3.2 11B,2024-09-24,Meta AI,United States of America,Industry,"Multimodal,Vision,Language",parameters,10600000000,10600000000,Confident,https://huggingface.co/meta-llama/Llama-3.2-11B-Vision,,,,Honeybee to mouse,5.79e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/,CC BY 4.0 (Epoch AI)
model,PixelDance,2024-09-24,ByteDance,China,Industry,"Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://pixeldance.io/ (LINK BROKEN),CC BY 4.0 (Epoch AI)
model,Movie Gen Video,2024-10-04,Meta AI,United States of America,Industry,"Video,Vision",parameters,30000000000,30000000000,Confident,30B,,,,Honeybee to mouse,1.65e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/static-resource/movie-gen-research-paper,CC BY 4.0 (Epoch AI)
model,GR-2,2024-10-08,ByteDance,China,Industry,Robotics,parameters,230000000,230000000,Confident,"the default GR-2 model contains 230M parameters, of which 95M are trainable",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2410.06158v1,CC BY 4.0 (Epoch AI)
model,Palmyra X 004,2024-10-09,Writer,United States of America,Industry,Language,parameters,150000000000,150000000000,,Source: https://venturebeat.com/ai/writers-palmyra-x-004-takes-the-lead-in-ai-function-calling-surpassing-tech-giants/,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://writer.com/engineering/actions-with-palmyra-x-004/,CC BY 4.0 (Epoch AI)
model,RDT-1B,2024-10-10,Tsinghua University,China,Academia,Robotics,parameters,1200000000,1200000000,Likely,"Model Training and Inference: ""We scale the size of RDT up to 1.2B parameters, establishing it as the currently largest diffusion-based robotic foundation model.""",,,,Honeybee to mouse,4.06e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2410.07864,CC BY 4.0 (Epoch AI)
model,Llama-3.1-Nemotron-70B-Instruct,2024-10-12,"Nvidia,Meta AI",United States of America,Industry,Language,parameters,,,Confident,,,,,,7.929e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://catalog.ngc.nvidia.com/orgs/nim/teams/nvidia/containers/llama-3.1-nemotron-70b-instruct,CC BY 4.0 (Epoch AI)
model,CHAI-1,2024-10-15,Chai discovery,United States of America,Industry,Biology,parameters,,,Confident,,,,,,7.7605724e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.chaidiscovery.com/blog/introducing-chai-1
https://www.biorxiv.org/content/10.1101/2024.10.10.615955v2",CC BY 4.0 (Epoch AI)
model,Yi-Lightning,2024-10-16,01.AI,China,Industry,Language,parameters,,,Confident,,,,,,1.5e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.lingyiwanwu.com/en https://platform.lingyiwanwu.com/,CC BY 4.0 (Epoch AI)
model,Claude 3.5 Sonnet (October 2024),2024-10-22,Anthropic,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/3-5-models-and-computer-use,CC BY 4.0 (Epoch AI)
model,NVLM-D 72B,2024-10-22,Nvidia,United States of America,Industry,"Vision,Language",parameters,72000000000,72000000000,Confident,72B,,,,Honeybee to mouse,3.02e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2409.11402,CC BY 4.0 (Epoch AI)
model,NVLM-H 72B,2024-10-22,Nvidia,United States of America,Industry,"Vision,Language",parameters,72000000000,72000000000,Likely,72B,,,,Honeybee to mouse,3.02e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2409.11402,CC BY 4.0 (Epoch AI)
model,NVLM-X 72B,2024-10-22,Nvidia,United States of America,Industry,"Vision,Language",parameters,72000000000,72000000000,Likely,72B,,,,Honeybee to mouse,3.0398181e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2409.11402,CC BY 4.0 (Epoch AI)
model,Doubao-pro,2024-10-28,ByteDance,China,Industry,Language,parameters,500000000000,500000000000,Likely,"[Speculative] Doubao's large language model has scaled up from 35 billion parameters to 800 billion, with 500 billion and 800 billion parameter models currently under training. https://xueqiu.com/9637001584/309910396?md5__1038=7qmx2DyDuie4cDBqDTQEWqDtMvO4iTphD ",,,,Mouse to cat,2.505e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.volcengine.com/docs/6360/1264663,CC BY 4.0 (Epoch AI)
model,Hunyuan-Large,2024-11-06,Tencent,China,Industry,Language,parameters,389000000000,389000000000,Confident,"""a total of 389 billion parameters and 52 billion activation parameters""",,,,Mouse to cat,3.49237e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2411.02265,CC BY 4.0 (Epoch AI)
model,SeedEdit,2024-11-11,ByteDance,China,Industry,Vision,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/SeedEdit.pdf
https://team.doubao.com/en/special/seededit",CC BY 4.0 (Epoch AI)
model,Gemini-Exp-1114,2024-11-15,Google DeepMind,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.tomsguide.com/ai/google-gemini/google-drops-new-gemini-model-and-it-goes-straight-to-the-top-of-the-llm-leaderboard,CC BY 4.0 (Epoch AI)
model,k0-math,2024-11-16,Moonshot,China,Industry,"Mathematics,Language",parameters,,,Unknown,This tweet mentions that this LLM has 100B params but I have not found the information anywhere else. https://x.com/kimmonismus/status/1902710969534460109,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://web.archive.org/web/20250124103542/https://www.globaltimes.cn/page/202411/1323248.shtml,CC BY 4.0 (Epoch AI)
model,Pixtral Large,2024-11-18,Mistral AI,France,Industry,"Multimodal,Language,Vision",parameters,124000000000,124000000000,Confident,"123B multimodal decoder, 1B parameter vision encoder",,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mistral.ai/news/pixtral-large/,CC BY 4.0 (Epoch AI)
model,Suno v4,2024-11-19,Suno,United States of America,Industry,Audio,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://suno.com/blog/v4,CC BY 4.0 (Epoch AI)
model,GPT-4o (Nov 2024),2024-11-20,OpenAI,United States of America,Industry,"Multimodal,Language,Audio,Speech,Vision",parameters,,,Speculative,Not known.  Inference costs in the API are 2x cheaper than GPT-4 Turbo,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://help.openai.com/en/articles/9624314-model-release-notes,CC BY 4.0 (Epoch AI)
model,Fugatto 1,2024-11-25,Nvidia,United States of America,Industry,"Multimodal,Language,Audio",parameters,2500000000,2500000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.nvidia.com/publication/2024-11_fugatto-1-foundational-generative-audio-transformer-opus-1,CC BY 4.0 (Epoch AI)
model,Amazon Nova Pro,2024-12-03,Amazon,United States of America,Industry,"Multimodal,Language,Video,Vision",parameters,,,Speculative,,,,,,6.000010000000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://aws.amazon.com/es/blogs/aws/introducing-amazon-nova-frontier-intelligence-and-industry-leading-price-performance/

https://assets.amazon.science/96/7d/0d3e59514abf8fdcfafcdc574300/nova-tech-report-20250317-0810.pdf",CC BY 4.0 (Epoch AI)
model,Infinity,2024-12-05,ByteDance,China,Industry,Image generation,parameters,2000000000,2000000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2412.04431v1,CC BY 4.0 (Epoch AI)
model,NVILA 15B,2024-12-05,"Nvidia,Massachusetts Institute of Technology (MIT),University of California (UC) Berkeley,University of California San Diego,University of Washington,Tsinghua University",China; United States of America,Mixed or other,"Vision,Language,Multimodal,Video",parameters,15000000000,15000000000,Confident,15B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2412.04468,CC BY 4.0 (Epoch AI)
model,o1,2024-12-05,OpenAI,United States of America,Industry,"Language,Mathematics,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-chatgpt-pro/,CC BY 4.0 (Epoch AI)
model,Llama 3.3 70B,2024-12-06,Meta AI,United States of America,Industry,Language,parameters,70000000000,70000000000,Confident,70B,,,,Honeybee to mouse,6.8649768e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_3/,CC BY 4.0 (Epoch AI)
model,EXAONE 3.5 32B,2024-12-09,LG AI Research,South Korea,Industry,Language,parameters,32000000000,32000000000,Confident,32B,,,,Honeybee to mouse,1.25e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2412.04862,CC BY 4.0 (Epoch AI)
model,Sora Turbo,2024-12-09,OpenAI,United States of America,Industry,"Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/sora-is-here/,CC BY 4.0 (Epoch AI)
model,Gemini 2.0 Pro,2024-12-11,Google DeepMind,United States of America,Industry,"Language,Multimodal,Vision,Video,Audio",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://deepmind.google/technologies/gemini/pro/,CC BY 4.0 (Epoch AI)
model,Apollo 7B,2024-12-13,"Meta AI,Stanford University",United States of America,Mixed or other,"Multimodal,Video,Language",parameters,7000000000,7000000000,Confident,"""We employed the Qwen2.5 (Yang et al., 2024) series of Large Language Models (LLMs) at varying scales to serve as the backbone for Apollo. Specifically, we utilized models with 1.5B, 3B, and 7B parameters. Following our analysis in Sec. 4, we used a SigLIP-SO400M (Zhai et al., 2023) encoder combined",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2412.10360,CC BY 4.0 (Epoch AI)
model,Veo 2,2024-12-16,Google DeepMind,United States of America,Industry,"Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://deepmind.google/technologies/veo/veo-2/,CC BY 4.0 (Epoch AI)
model,o3,2024-12-20,OpenAI,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-o3-and-o4-mini/,CC BY 4.0 (Epoch AI)
model,DeepSeek-V3,2024-12-24,DeepSeek,China,Industry,Language,parameters,671000000000,671000000000,Confident,Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.,yes,,,Mouse to cat,3.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2412.19437,CC BY 4.0 (Epoch AI)
model,STORM-B/8,2024-12-31,"University of Southern California,Georgia Institute of Technology,Stanford University,Nvidia",United States of America,Mixed or other,3D modeling,parameters,100598707,100598707,Confident,"A.3 Baseline Implementations: ""The number of trainable parameters of these models are controlled to be similar, i.e., GS-LRM has 86.68M parameters, LGM has 103.29M parameters, while our default STORM has 100.60M parameters.  3.3 Implementation, Model architecture: ""By default, we use a 12-layer Visi",,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2501.00602
https://github.com/NVlabs/GaussianSTORM
https://jiawei-yang.github.io/STORM",CC BY 4.0 (Epoch AI)
model,Eurus-2-7B-PRIME,2025-01-02,"Tsinghua University,University of Illinois Urbana-Champaign (UIUC),Shanghai AI Lab,Peking University,Shanghai Jiao Tong University,CUHK Shenzhen Research Institute",China; United States of America,Academia,"Mathematics,Language",parameters,7000000000,7000000000,Speculative,,,yes,,Honeybee to mouse,4.01915904e+20,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2502.01456,CC BY 4.0 (Epoch AI)
model,Stable Point Aware 3D (SPAR3D),2025-01-08,"Stability AI,University of Illinois Urbana-Champaign (UIUC)",United Kingdom; United States of America,Mixed or other,3D modeling,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2501.04689,CC BY 4.0 (Epoch AI)
model,INTELLECT-MATH,2025-01-17,Prime Intellect,United States of America,Industry,Mathematics,parameters,7000000000,7000000000,Confident,,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/PrimeIntellect/INTELLECT-MATH
https://www.primeintellect.ai/blog/intellect-math",CC BY 4.0 (Epoch AI)
model,DeepSeek-R1,2025-01-20,DeepSeek,China,Industry,Language,parameters,671000000000,671000000000,Confident,671B total 37B activated https://github.com/deepseek-ai/DeepSeek-R1/tree/main,yes,,,Mouse to cat,3.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://api-docs.deepseek.com/news/news250120,CC BY 4.0 (Epoch AI)
model,Eagle 2,2025-01-20,"Nvidia,Nanjing University,Tsinghua University,Hong Kong Polytechnic University,Johns Hopkins University,New York University (NYU)",China; Hong Kong; United States of America,Mixed or other,"Vision,Robotics,Language",parameters,8930000000,8930000000,Confident,"Table 4  https://huggingface.co/nvidia/Eagle2-9B ""8.93B params"" ",,,,Honeybee to mouse,4.7156e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,arxiv.org/abs/2501.14818,CC BY 4.0 (Epoch AI)
model,Doubao-1.5-pro,2025-01-22,ByteDance,China,Industry,Language,parameters,,,Confident,"Not directly reported. We are told it is a MoE model, and that it matches the performance of a dense model trained on the same data, while using 1/7th of the activated parameters. Additionally they say ""The number of parameters of the Doubao dense model is also much smaller than that of Llama3.1-405",yes,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://team.doubao.com/zh/special/doubao_1_5_pro,CC BY 4.0 (Epoch AI)
model,Kimi k1.5,2025-01-22,Moonshot,China,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2501.12599v1,CC BY 4.0 (Epoch AI)
model,Computer-Using Agent (CUA),2025-01-23,OpenAI,United States of America,Industry,"Vision,Language,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/computer-using-agent/,CC BY 4.0 (Epoch AI)
model,GPT-4o (Jan 2025),2025-01-29,OpenAI,United States of America,Industry,"Multimodal,Language,Audio,Speech,Vision",parameters,,,Speculative,Not known.  Inference costs in the API are 2x cheaper than GPT-4 Turbo,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://help.openai.com/en/articles/9624314-model-release-notes,CC BY 4.0 (Epoch AI)
model,o3-mini,2025-01-31,OpenAI,United States of America,Industry,Language,parameters,,,Unknown,"Can't get an exact estimate, but we suspect total parameter count around 60B-120B, active parameters around 10B-30B.   Given these models are served at 150-200 tok/s, at $4.40/Mtok output, inference economics (https://epoch.ai/blog/inference-economics-of-language-models) suggests total parameter cou",yes,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/openai-o3-mini/,CC BY 4.0 (Epoch AI)
model,Grok 3,2025-02-17,xAI,United States of America,Industry,"Language,Vision,Multimodal",parameters,3000000000000,3000000000000,Likely,,,,,Cat or more,3.5e+26,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/blog/grok-3,CC BY 4.0 (Epoch AI)
model,Claude 3.7 Sonnet,2025-02-24,Anthropic,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Likely,,,,,,3.35e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-3-7-sonnet,CC BY 4.0 (Epoch AI)
model,GPT-4.5,2025-02-27,OpenAI,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Likely,,,,,,3.8e+26,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-4-5/,CC BY 4.0 (Epoch AI)
model,Mercury,2025-02-27,Inception Labs,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://machine-learning-made-simple.medium.com/is-the-mercury-llm-the-first-of-a-new-generation-of-llms-b64de1d36029

https://www.inceptionlabs.ai/news

https://www.inceptionlabs.ai/introducing-mercury-our-general-chat-model
",CC BY 4.0 (Epoch AI)
model,Mistral OCR,2025-03-06,Mistral AI,France,Industry,"Multimodal,Vision,Language",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mistral.ai/news/mistral-ocr,CC BY 4.0 (Epoch AI)
model,QwQ-32B,2025-03-06,Alibaba,China,Industry,Language,parameters,32500000000,32500000000,Speculative,"Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias Number of Parameters: 32.5B Number of Paramaters (Non-Embedding): 31.0B Number of Layers: 64 Number of Attention Heads (GQA): 40 for Q and 8 for KV",,yes,,Honeybee to mouse,3.51e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwq-32b/,CC BY 4.0 (Epoch AI)
model,Hunyuan-TurboS,2025-03-11,Tencent,China,Industry,Language,parameters,560000000000,560000000000,Confident, the model scales to 56B activated parameters and 560B total parameters,yes,,,Mouse to cat,5.4e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://web.archive.org/web/20250408105622/https://www.dapingtime.com/article/2171.html

https://medium.com/data-science-in-your-pocket/tencent-hunyuan-turbo-s-the-fastest-reasoning-llm-d64a02bed5c8

https://arxiv.org/abs/2505.15431",CC BY 4.0 (Epoch AI)
model,ERNIE-4.5-VL-424B-A47B (文心大模型4.5),2025-03-16,Baidu,China,Industry,"Multimodal,Language,Vision,Video",parameters,424000000000,424000000000,Confident,"MoE: total parameters - 424B active parameters - 47B  ""ViT encoder comprising 630 million parameters""",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.datacamp.com/blog/ernie-4-5-x1

https://yiyan.baidu.com/blog/publication/ERNIE_Technical_Report.pdf",CC BY 4.0 (Epoch AI)
model,EXAONE Deep 32B,2025-03-16,LG AI Research,South Korea,Industry,Language,parameters,32000000000,32000000000,Confident,32B,,,,Honeybee to mouse,1.26e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2503.12524,CC BY 4.0 (Epoch AI)
model,o1-pro,2025-03-19,OpenAI,United States of America,Industry,"Language,Mathematics,Multimodal",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-chatgpt-pro/,CC BY 4.0 (Epoch AI)
model,Diffusion Renderer,2025-03-22,"Nvidia,University of Toronto,Vector Institute,University of Illinois Urbana-Champaign (UIUC)",Canada; United States of America,Mixed or other,Video,parameters,1100000000,1100000000,Confident,"""This model has 1.1B model parameters.""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2501.18590,CC BY 4.0 (Epoch AI)
model,DeepSeek-V3 (Mar 2025),2025-03-24,DeepSeek,China,Industry,Language,parameters,671000000000,671000000000,Confident,Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.,yes,,,Mouse to cat,3.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2412.19437,CC BY 4.0 (Epoch AI)
model,Gemini 2.5 Pro (Mar 2025),2025-03-25,Google DeepMind,United States of America,Industry,"Language,Vision,Video,Multimodal,Speech",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/#gemini-2-5-thinking,CC BY 4.0 (Epoch AI)
model,GPT-4o (Mar 2025),2025-03-27,OpenAI,United States of America,Industry,"Multimodal,Language,Audio,Speech,Vision",parameters,,,Speculative,Not known.  Inference costs in the API are 2x cheaper than GPT-4 Turbo,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://help.openai.com/en/articles/6825453-chatgpt-release-notes,CC BY 4.0 (Epoch AI)
model,Llama 4 Behemoth (preview),2025-04-05,Meta AI,United States of America,Industry,"Multimodal,Language,Vision",parameters,2000000000000,2000000000000,Likely,"""Llama 4 Behemoth, a 288 billion active parameter model with 16 experts that is our most powerful yet and among the world’s smartest LLMs.""",yes,,,Mouse to cat,5.18400000000001e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/blog/llama-4-multimodal-intelligence/,CC BY 4.0 (Epoch AI)
model,Llama 4 Maverick,2025-04-05,Meta AI,United States of America,Industry,"Multimodal,Language,Vision",parameters,400000000000,400000000000,Likely,"""Llama 4 Maverick models have 17B active parameters and 400B total parameters.""  https://ai.meta.com/blog/llama-4-multimodal-intelligence/",yes,,,Mouse to cat,2.244000000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/blog/llama-4-multimodal-intelligence/,CC BY 4.0 (Epoch AI)
model,Llama 4 Scout,2025-04-05,Meta AI,United States of America,Industry,"Multimodal,Language,Vision",parameters,109000000000,109000000000,Likely,"""Our smaller model, Llama 4 Scout, is a general purpose model with 17 billion active parameters, 16 experts, and 109 billion total parameters that delivers state-of-the-art performance for its class.""",yes,,,Mouse to cat,4.08e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/blog/llama-4-multimodal-intelligence/,CC BY 4.0 (Epoch AI)
model,Pangu Ultra,2025-04-10,Huawei,China,Industry,Language,parameters,135000000000,135000000000,Confident,,,,,Mouse to cat,1.0692e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2504.07866,CC BY 4.0 (Epoch AI)
model,gpt-image-1,2025-04-23,OpenAI,United States of America,Industry,"Image generation,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/image-generation-api/,CC BY 4.0 (Epoch AI)
model,Qwen3-235B-A22B,2025-04-29,Alibaba,China,Industry,Language,parameters,235000000000,235000000000,Likely,"235 billion total parameters and 22 billion activated parameters  Number of Layers: 94 Number of Attention Heads (GQA): 64 for Q and 4 for KV Number of Experts: 128 Number of Activated Experts: 8 Context Length: 32,768 natively and 131,072 tokens with YaRN.",yes,,,Mouse to cat,4.752e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen3/,CC BY 4.0 (Epoch AI)
model,Gemini 2.5 Pro (May 2025),2025-05-06,Google DeepMind,United States of America,Industry,"Language,Vision,Video,Multimodal,Speech",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/#gemini-2-5-thinking,CC BY 4.0 (Epoch AI)
model,Seed1.5-VL,2025-05-11,ByteDance,China,Industry,"Vision,Language,Multimodal,Video",parameters,,,Confident,"""Seed1.5-VL is composed with a 532M-parameter vision encoder and a Mixture-of-Experts (MoE) LLM of 20B active parameters.""",yes,,,,1.388556e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2505.07062,CC BY 4.0 (Epoch AI)
model,Imagen 4,2025-05-20,Google,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://storage.googleapis.com/deepmind-media/Model-Cards/Imagen-4-Model-Card.pdf,CC BY 4.0 (Epoch AI)
model,Veo 3,2025-05-21,Google DeepMind,United States of America,Industry,"Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://deepmind.google/models/veo/,CC BY 4.0 (Epoch AI)
model,Claude Opus 4,2025-05-22,Anthropic,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.anthropic.com/claude/opus

https://www.anthropic.com/news/claude-4",CC BY 4.0 (Epoch AI)
model,Claude Sonnet 4,2025-05-22,Anthropic,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www.anthropic.com/claude/sonnet

https://www.anthropic.com/news/claude-4",CC BY 4.0 (Epoch AI)
model,DeepSeek-R1 (May 2025),2025-05-28,DeepSeek,China,Industry,Language,parameters,671000000000,671000000000,Confident,671B total 37B activated https://github.com/deepseek-ai/DeepSeek-R1/tree/main,yes,,,Mouse to cat,3.3e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://api-docs.deepseek.com/news/news250120,CC BY 4.0 (Epoch AI)
model,Gemini 2.5 Pro (Jun 2025),2025-06-05,Google DeepMind,United States of America,Industry,"Language,Vision,Video,Multimodal,Speech",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/#gemini-2-5-thinking,CC BY 4.0 (Epoch AI)
model,Qwen3 Embedding,2025-06-05,Alibaba,China,Industry,Language,parameters,8000000000,8000000000,Confident,8B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2506.05176v1,CC BY 4.0 (Epoch AI)
model,Seed-1.6-Thinking,2025-06-11,ByteDance,China,Industry,"Multimodal,Language,Vision",parameters,230000000000,230000000000,Confident,"Model has 230B parameters total, with 23B active parameters (sparse MoE setting).",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://seed.bytedance.com/en/blog/introduction-to-techniques-used-in-seed1-6,CC BY 4.0 (Epoch AI)
model,FGN,2025-06-12,Google DeepMind,United States of America,Industry,Earth science,parameters,720000000,720000000,Confident,"""In this work we train 𝐽 = 4 models. To ensemble their predictions, we generate an equal number of ensemble member trajectories from each model"" ""FGN is a larger model, with ∼180m parameters per model seed""  They train a 4 model ensemble with 180M parameters per model. Total parameters: 4*180=720M",,,,Honeybee to mouse,9.618950880000001e+21,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2506.10772v1,CC BY 4.0 (Epoch AI)
model,EXAONE Path 2.0,2025-07-09,LG AI Research,South Korea,Industry,"Vision,Medicine",parameters,175000000,175000000,Confident,~175M parameters based on Figure 1,,,,Fruit fly to honeybee,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2507.06639,CC BY 4.0 (Epoch AI)
model,Grok 4,2025-07-09,xAI,United States of America,Industry,"Language,Multimodal,Vision",parameters,3000000000000,3000000000000,Speculative,Rumored to be 2.4T params (https://x.com/kalomaze/status/1942996555088134592),,yes,,Cat or more,5.0000000000001e+26,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/news/grok-4,CC BY 4.0 (Epoch AI)
model,Grok 4 Heavy,2025-07-10,xAI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/news/grok-4,CC BY 4.0 (Epoch AI)
model,Kimi K2,2025-07-11,Moonshot,China,Industry,Language,parameters,1000000000000,1000000000000,Confident,MoE with 1T total parameters and 32B parameters active per forward pass,yes,,,Mouse to cat,2.976e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://moonshotai.github.io/Kimi-K2/,CC BY 4.0 (Epoch AI)
model,Gemini Embedding,2025-07-14,Google DeepMind,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2503.07891,CC BY 4.0 (Epoch AI)
model,EXAONE 4.0 (32B),2025-07-15,LG AI Research,South Korea,Industry,Language,parameters,32000000000,32000000000,Confident,32B,,,,Honeybee to mouse,2.69000000000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2507.11407,CC BY 4.0 (Epoch AI)
model,Qwen3-Coder-480B-A35B,2025-07-22,Alibaba,China,Industry,Language,parameters,480000000000,480000000000,Confident,"""a 480B-parameter Mixture-of-Experts model with 35B active parameters which supports the context length of 256K tokens natively and 1M tokens with extrapolation methods""",yes,,,Mouse to cat,1.575e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen3-coder/,CC BY 4.0 (Epoch AI)
model,Qwen3-235B-A22B (Jul 2025),2025-07-25,Alibaba,China,Industry,Language,parameters,235000000000,235000000000,Likely,"235 billion total parameters and 22 billion activated parameters  Number of Layers: 94 Number of Attention Heads (GQA): 64 for Q and 4 for KV Number of Experts: 128 Number of Activated Experts: 8 Context Length: 32,768 natively and 131,072 tokens with YaRN.",yes,,,Mouse to cat,4.752e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen3/,CC BY 4.0 (Epoch AI)
model,Qwen3-235B-A22B-Thinking (Jul 2025),2025-07-25,Alibaba,China,Industry,Language,parameters,235000000000,235000000000,Likely,"235 billion total parameters and 22 billion activated parameters  Number of Layers: 94 Number of Attention Heads (GQA): 64 for Q and 4 for KV Number of Experts: 128 Number of Activated Experts: 8 Context Length: 32,768 natively and 131,072 tokens with YaRN.",yes,,,Mouse to cat,4.752e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwenlm.github.io/blog/qwen3/,CC BY 4.0 (Epoch AI)
model,GLM-4.5,2025-07-28,"Z.ai (Zhipu AI),Tsinghua University",China,Mixed or other,Language,parameters,355000000000,355000000000,Confident,355 billion total parameters with 32 billion active parameters,yes,,,Mouse to cat,4.42e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2508.06471,CC BY 4.0 (Epoch AI)
model,Gemini 2.5 Deep Think,2025-08-01,"Google,Google DeepMind",United States of America,Industry,"Language,Multimodal,Vision,Video,Audio,Mathematics",parameters,,,Unknown,"The Gemini 2.5 models are sparse mixture-of-experts (MoE) transformers with native multimodal support for text, vision, and audio inputs.",yes,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-2-5-Deep-Think-Model-Card.pdf

https://storage.googleapis.com/deepmind-media/gemini/gemini_v2_5_report.pdf",CC BY 4.0 (Epoch AI)
model,MindLink-72B,2025-08-01,Kunlun Inc.,China,Industry,Language,parameters,72000000000,72000000000,Confident,72B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://github.com/SkyworkAI/MindLink/blob/main/mindlink.pdf,CC BY 4.0 (Epoch AI)
model,Hierarchical Reasoning Model (HPM),2025-08-04,Sapient Intelligence,Singapore,Not stated,"Language,Vision,Multimodal",parameters,27000000,27000000,Confident,27M,,,,Below a fruit fly,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2506.21734,CC BY 4.0 (Epoch AI)
model,Qwen Image,2025-08-04,Alibaba,China,Industry,"Image generation,Vision",parameters,27000000000,27000000000,Confident,Table 1,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/Qwen_Image.pdf,CC BY 4.0 (Epoch AI)
model,Claude Opus 4.1,2025-08-05,Anthropic,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-opus-4-1,CC BY 4.0 (Epoch AI)
model,gpt-oss-120b,2025-08-05,OpenAI,United States of America,Industry,Language,parameters,116830000000,116830000000,Confident,Total parameters: 116.83B,,,,Mouse to cat,4.94e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf,CC BY 4.0 (Epoch AI)
model,gpt-oss-20b,2025-08-05,OpenAI,United States of America,Industry,Language,parameters,20910000000,20910000000,Confident,Total parameters: 20.91B,,,,Honeybee to mouse,5.49e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf,CC BY 4.0 (Epoch AI)
model,GPT-5,2025-08-07,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Speculative,,,,,,6.6e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://cdn.openai.com/gpt-5-system-card.pdf
https://openai.com/index/introducing-gpt-5/ ",CC BY 4.0 (Epoch AI)
model,GPT-5 mini,2025-08-07,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/gpt-5-system-card.pdf,CC BY 4.0 (Epoch AI)
model,GPT-5 nano,2025-08-07,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/gpt-5-system-card.pdf,CC BY 4.0 (Epoch AI)
model,Gemini 2.5 Flash Image (Nano Banana),2025-08-26,Google,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://gemini.google/overview/image-generation/

https://aistudio.google.com/models/gemini-2-5-flash-image",CC BY 4.0 (Epoch AI)
model,LongCat-Flash,2025-09-01,Meituan Inc,China,Industry,Language,parameters,560000000000,560000000000,Likely,"""560 billion total parameters, featuring an innovative Mixture-of-Experts (MoE) architecture. The model incorporates a dynamic computation mechanism that activates 18.6B∼31.3B parameters (averaging∼27B)""",yes,,,Mouse to cat,3.726e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2509.01322,CC BY 4.0 (Epoch AI)
model,Qwen3-Max,2025-09-05,Alibaba,China,Industry,Language,parameters,1000000000000,1000000000000,Speculative,MoE architecture,yes,yes,,Mouse to cat,1.512e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://modelstudio.console.alibabacloud.com/?tab=doc#/doc/?type=model&url=2840914_2&modelId=qwen3-max-preview

https://qwen.ai/blog?id=87dc93fc8a590dc718c77e1f6e84c07b474f6c5a&from=home.latest-research-list",CC BY 4.0 (Epoch AI)
model,AgentFounder-30B,2025-09-16,Alibaba,China,Industry,Language,parameters,30000000000,30000000000,Confident,30B-A3B,,,,Honeybee to mouse,6.5367e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2509.13310,CC BY 4.0 (Epoch AI)
model,Qwen3-Omni-30B-A3B,2025-09-22,Alibaba,China,Industry,"Multimodal,Language,Vision,Speech,Video",parameters,35300000000,35300000000,Confident,Audio Encoder AuT 650M  Vision Encoder SigLIP2-So400M 540M  Thinker MoE Transformer 30B-A3B Talker MoE Transformer 3B-A0.3B MTP Dense Transformer 80M Code2wav ConvNet 200M,yes,,,Honeybee to mouse,3.6e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2509.17765,CC BY 4.0 (Epoch AI)
model,Qwen3-VL-235B-A22B,2025-09-23,Alibaba,China,Industry,"Language,Vision,Multimodal,Video",parameters,235000000000,235000000000,Likely,"235B total, 22B active",yes,,,Mouse to cat,5.038044e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2511.21631,CC BY 4.0 (Epoch AI)
model,Gemini Robotics-ER 1.5,2025-09-25,Google DeepMind,United States of America,Industry,"Vision,Language,Speech",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://deepmind.google/discover/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/

https://storage.googleapis.com/deepmind-media/gemini-robotics/Gemini-Robotics-1-5-Tech-Report.pdf",CC BY 4.0 (Epoch AI)
model,Claude Sonnet 4.5,2025-09-29,Anthropic,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-sonnet-4-5,CC BY 4.0 (Epoch AI)
model,GLM-4.6,2025-09-30,"Z.ai (Zhipu AI),Tsinghua University",China,Mixed or other,Language,parameters,357000000000,357000000000,Likely,Similarly to GLM 4.5: 355 billion total parameters (reported) with 32 billion active parameters (assumed),yes,,,Mouse to cat,4.42e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://z.ai/blog/glm-4.6,CC BY 4.0 (Epoch AI)
model,Sora 2.0,2025-09-30,OpenAI,United States of America,Industry,Video,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/sora-2/,CC BY 4.0 (Epoch AI)
model,GPT-5 Pro,2025-10-07,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cdn.openai.com/gpt-5-system-card.pdf,CC BY 4.0 (Epoch AI)
model,Ling-1T,2025-10-10,Ant Group,China,Industry,Language,parameters,1000000000000,1000000000000,Confident,1 trillion total parameters with 50 billion activated parameters,yes,,,Mouse to cat,6.000001e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/inclusionAI/Ling-1T
https://arxiv.org/pdf/2510.22115",CC BY 4.0 (Epoch AI)
model,Claude Haiku 4.5,2025-10-15,Anthropic,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-haiku-4-5,CC BY 4.0 (Epoch AI)
model,Veo 3.1,2025-10-15,Google DeepMind,United States of America,Industry,"Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/technology/ai/veo-updates-flow/,CC BY 4.0 (Epoch AI)
model,MiniMax-M2,2025-10-27,MiniMax,China,Industry,Language,parameters,229000000000,229000000000,Confident,"""maintaining activations around 10B"" safetensors: 229B params",,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.minimax.io/news/minimax-m2,CC BY 4.0 (Epoch AI)
model,Tongyi DeepResearch,2025-10-28,Alibaba,China,Industry,Language,parameters,30500000000,30500000000,Confident,"30.5 billion total parameters, with only 3.3 billion activated per token",yes,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2510.24701,CC BY 4.0 (Epoch AI)
model,Kimi K2 Thinking,2025-11-06,Moonshot,China,Industry,Language,parameters,1000000000000,1000000000000,Likely,Total Parameters	1T Activated Parameters	32B,yes,,,Mouse to cat,4.2e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://moonshotai.github.io/Kimi-K2/thinking,CC BY 4.0 (Epoch AI)
model,Kaiju Large,2025-11-07,Character.ai,United States of America,Industry,Language,parameters,110000000000,110000000000,Likely,"""The Kaiju family of models comes in 3 production variants: Small (13B), Medium (34B), and Large (110B).""",,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.character.ai/inside-kaiju-building-conversational-models-at-scale/ ,CC BY 4.0 (Epoch AI)
model,Kaiju Medium,2025-11-07,Character.ai,United States of America,Industry,Language,parameters,34000000000,34000000000,Likely,"""The Kaiju family of models comes in 3 production variants: Small (13B), Medium (34B), and Large (110B).""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.character.ai/inside-kaiju-building-conversational-models-at-scale/ ,CC BY 4.0 (Epoch AI)
model,Kaiju Small,2025-11-07,Character.ai,United States of America,Industry,Language,parameters,13000000000,13000000000,Likely,"""The Kaiju family of models comes in 3 production variants: Small (13B), Medium (34B), and Large (110B).""",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.character.ai/inside-kaiju-building-conversational-models-at-scale/ ,CC BY 4.0 (Epoch AI)
model,Meta's Generative Ads Model (GEM),2025-11-10,Meta AI,United States of America,Industry,Recommendation,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://engineering.fb.com/2025/11/10/ml-applications/metas-generative-ads-model-gem-the-central-brain-accelerating-ads-recommendation-ai-innovation/,CC BY 4.0 (Epoch AI)
model,GPT-5.1-Codex,2025-11-12,OpenAI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,,CC BY 4.0 (Epoch AI)
model,GPT-5.1,2025-11-13,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/gpt-5-1/,CC BY 4.0 (Epoch AI)
model,GPT-5.1 Instant,2025-11-13,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/gpt-5-1/,CC BY 4.0 (Epoch AI)
model,Grok 4.1,2025-11-17,xAI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/news/grok-4-1,CC BY 4.0 (Epoch AI)
model,P1-235B-A22B,2025-11-17,Shanghai AI Lab,China,Academia,Language,parameters,235000000000,235000000000,Likely,"MoE: P1-235B-A22B has 235B total, 22B active parameters. Lightweight variant P1-30B-A3B has 30B total, 3B active. (https://arxiv.org/abs/2511.13612)",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2511.13612, https://github.com/PRIME-RL/P1",CC BY 4.0 (Epoch AI)
model,π0.6 (pi-0.6),2025-11-17,Physical Intelligence,United States of America,Industry,"Robotics,Vision",parameters,5260000000,5260000000,Confident,"""(ii) The base VLM is Gemma 3 [78] 4B model. (iii) The size of the action expert is increased to 860M parameters."" Model card (https://website.pi-asset.com/pi06star/PI06_model_card.pdf) notes in Figure 1 that the VLM also includes SigLIP (400M). ",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.pi.website/download/pistar06.pdf,CC BY 4.0 (Epoch AI)
model,Gemini 3 Deep Think,2025-11-18,Google DeepMind,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-deep-think/,CC BY 4.0 (Epoch AI)
model,Gemini 3 Pro,2025-11-18,Google DeepMind,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/products/gemini/gemini-3/,CC BY 4.0 (Epoch AI)
model,GPT-5.1-Codex-Max,2025-11-19,OpenAI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/gpt-5-1-codex-max/,CC BY 4.0 (Epoch AI)
model,Grok 4.1 Fast,2025-11-19,xAI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/news/grok-4-1-fast,CC BY 4.0 (Epoch AI)
model,Gemini 3 Pro Image (Nano Banana Pro),2025-11-20,Google DeepMind,United States of America,Industry,Image generation,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/innovation-and-ai/products/nano-banana-pro/,CC BY 4.0 (Epoch AI)
model,Olmo 3,2025-11-20,Allen Institute for AI,United States of America,Mixed or other,Language,parameters,32000000000,32000000000,Confident,32B with Base/Instruct/Think variants. Base here,,,,Honeybee to mouse,1.1e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2512.13961,CC BY 4.0 (Epoch AI)
model,Claude Opus 4.5,2025-11-24,Anthropic,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-opus-4-5,CC BY 4.0 (Epoch AI)
model,DeepSeekMath-V2,2025-11-27,DeepSeek,China,Industry,Language,parameters,685000000000,685000000000,Confident,HF lists 685B params. Built on top of DeepSeek-V3.2-Exp-Base,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2511.22570,CC BY 4.0 (Epoch AI)
model,SIMA 2,2025-12-04,Google DeepMind,United States of America,Industry,3D modeling,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2512.04797v1,CC BY 4.0 (Epoch AI)
model,GPT-5.2,2025-12-11,OpenAI,United States of America,Industry,,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,,CC BY 4.0 (Epoch AI)
model,GPT-5.2 Pro,2025-12-11,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-2/,CC BY 4.0 (Epoch AI)
model,Nemotron 3-Nano-30B-A3B,2025-12-15,Nvidia,United States of America,Industry,Language,parameters,31600000000,31600000000,Likely,"""Nano 30B-A3B Base contains 31.6B total parameters out of which 3.2B are activ""",,,,Honeybee to mouse,4.8e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf,CC BY 4.0 (Epoch AI)
model,GPT Image 1.5,2025-12-16,OpenAI,United States of America,Industry,"Image generation,Language",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/new-chatgpt-images-is-here/,CC BY 4.0 (Epoch AI)
model,GPT-5.2 Codex,2025-12-18,OpenAI,United States of America,Industry,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-2-codex/,CC BY 4.0 (Epoch AI)
model,GLM-4.7,2025-12-22,Z.ai (Zhipu AI),China,Industry,Language,parameters,358000000000,358000000000,Likely,"mixture of experts (MoE) with 358B total, 32B active",yes,,,Mouse to cat,4.42e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://z.ai/blog/glm-4.7,CC BY 4.0 (Epoch AI)
model,MiniMax-M2.1,2025-12-23,MiniMax,China,Industry,Language,parameters,229000000000,229000000000,Confident,HF lists 229B params 10B active parameters,yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/MiniMaxAI/MiniMax-M2.1,CC BY 4.0 (Epoch AI)
model,HyperCLOVA X SEED 32B Think,2025-12-29,NAVER,South Korea,Industry,"Multimodal,Language,Vision",parameters,32000000000,32000000000,Confident,32B,,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/naver-hyperclovax/HyperCLOVAX-SEED-Think-32B,CC BY 4.0 (Epoch AI)
model,A.X K1,2025-12-30,SK Telecom,South Korea,Industry,Language,parameters,519000000000,519000000000,Confident,"MoE Architecture, Total parameters 519B, Active parameters 33B",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2601.09200 

https://huggingface.co/skt/A.X-K1 ",CC BY 4.0 (Epoch AI)
model,VAETKI,2025-12-30,NC AI,South Korea,Industry,Language,parameters,100000000000,100000000000,Confident,"Total parameters: 100B (Sparse), Active parameters: 10B per token. It uses a Mixture-of-Experts architecture with 1 shared expert and 128 total experts(top-8 routing)",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/NC-AI-consortium-VAETKI/VAETKI,CC BY 4.0 (Epoch AI)
model,K-EXAONE,2025-12-31,LG AI Research,South Korea,Industry,Language,parameters,236000000000,236000000000,Confident,236B,,,,Mouse to cat,1.52e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/LGAI-EXAONE,CC BY 4.0 (Epoch AI)
model,Solar Open 100B,2025-12-31,Upstage,South Korea,Industry,Language,parameters,102000000000,102000000000,Confident,"102B, of which 12B active ",yes,,,Mouse to cat,1.4e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/upstage/Solar-Open-100B,CC BY 4.0 (Epoch AI)
model,Qwen3-Max-Thinking,2026-01-25,Alibaba,China,Industry,Language,parameters,,,Likely,Unknown,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwen.ai/blog?id=qwen3-max-thinking,CC BY 4.0 (Epoch AI)
model,Kimi K2.5,2026-02-02,Moonshot,China,Industry,Language,parameters,1040000000000,1040000000000,Likely,"""The model comprises 1.04 trillion total parameters""",,,,Mouse to cat,5.8e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2602.02276,CC BY 4.0 (Epoch AI)
model,Qwen3-Coder-Next,2026-02-02,Alibaba,China,Industry,Language,parameters,80000000000,80000000000,Likely,"Finetune of Qwen3-Next-80B-A3B, which has 80 billion parameters",,,,Honeybee to mouse,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwen.ai/blog?id=qwen3-coder-next,CC BY 4.0 (Epoch AI)
model,GPT-5.3 Codex,2026-02-05,OpenAI,United States of America,Industry,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-3-codex/,CC BY 4.0 (Epoch AI)
model,GLM-5,2026-02-12,Z.ai (Zhipu AI),China,Industry,Language,parameters,744000000000,744000000000,Likely,"""GLM-5 scales to 256 experts and reduces its layer count to 80 to minimize expert parallelism communication overhead. This results in a 744B parameter model""",,,,Mouse to cat,6.84e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/pdf/2602.15763 

https://z.ai/blog/glm-5 ",CC BY 4.0 (Epoch AI)
model,GPT-5.3-Codex-Spark,2026-02-12,OpenAI,United States of America,Industry,Language,parameters,,,Unknown,"Rumoured by SemiAnalysis (https://newsletter.semianalysis.com/p/cerebras-faster-tokens-please), who may have non-public knowledge, to be of the same size and architecture as GPT-OSS 120B. No public corroboration fo this as of September 2026.",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-3-codex-spark/,CC BY 4.0 (Epoch AI)
model,Seedance 2.0,2026-02-12,ByteDance,China,Industry,"Image generation,Video,Audio",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://seed.bytedance.com/en/seedance2_0,CC BY 4.0 (Epoch AI)
model,Doubao-Seed-2.0 Pro,2026-02-14,ByteDance,China,Industry,"Language,Vision,Multimodal,Video",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf

https://seed.bytedance.com/en/blog/seed2-0-%E6%AD%A3%E5%BC%8F%E5%8F%91%E5%B8%83",CC BY 4.0 (Epoch AI)
model,Qwen3.5 397B-A17B,2026-02-15,Alibaba,China,Industry,"Language,Vision",parameters,397000000000,397000000000,Likely,"""it comprises 397 billion total parameters""",,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwen.ai/blog?id=qwen3.5,CC BY 4.0 (Epoch AI)
model,Grok 4.20,2026-02-17,xAI,United States of America,Industry,Language,parameters,500000000000,500000000000,Likely,"""This is just our V8 small foundation model, so 500B params"" from Elon Musk on X (https://x.com/elonmusk/status/2023840935285760142?s=20)  Likely sparse",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://docs.x.ai/developers/models,CC BY 4.0 (Epoch AI)
model,Qwen3.5-122B-A10B,2026-02-24,Alibaba,China,Industry,Language,parameters,122000000000,122000000000,Likely,,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://github.com/QwenLM/Qwen3.5?tab=readme-ov-file ,CC BY 4.0 (Epoch AI)
model,SWE 1.6,2026-03-01,Cognition,United States of America,Industry,Language,parameters,,,Likely,"From SWE 1.5 blog post:  ""It is a frontier-size model with hundreds of billions of parameters.""",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cognition.ai/blog/swe-1-6-preview,CC BY 4.0 (Epoch AI)
model,Gemini 3.0 Flash-lite,2026-03-03,Google DeepMind,United States of America,Industry,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-flash-lite/,CC BY 4.0 (Epoch AI)
model,GPT-5.4,2026-03-05,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-4/,CC BY 4.0 (Epoch AI)
model,GPT-5.4 Pro,2026-03-05,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-4/,CC BY 4.0 (Epoch AI)
model,Nemotron 3 Super,2026-03-11,Nvidia,United States of America,Industry,Language,parameters,120000000000,120000000000,Likely,"""The new Super model is a 120B total, 12B active-parameter""",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://developer.nvidia.com/blog/introducing-nemotron-3-super-an-open-hybrid-mamba-transformer-moe-for-agentic-reasoning/,CC BY 4.0 (Epoch AI)
model,MiMo-V2-Pro,2026-03-18,Xiaomi Corp,China,Industry,Language,parameters,1020000000000,1020000000000,Confident,"""Total parameters surpass 1T with 42B active, roughly 3x larger than MiMo-V2-Flash."" Listed on https://mimo.xiaomi.com/mimo-v2-5-pro/ specifically as 1.02T in the ""Full benchmark results"" table",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mimo.xiaomi.com/mimo-v2-pro,CC BY 4.0 (Epoch AI)
model,Composer 2,2026-03-19,Cursor,United States of America,Industry,Language,parameters,1040000000000,1040000000000,Confident,"Based on Kimi K2.5; 1.04T total, 32B active parameters",yes,,,Mouse to cat,2.32e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2603.24477,CC BY 4.0 (Epoch AI)
model,Qwen3.5-Omni-Flash,2026-03-29,Alibaba,China,Industry,"Language,Multimodal,Vision,Audio",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2604.15804,CC BY 4.0 (Epoch AI)
model,Qwen3.5-Omni-Plus,2026-03-29,Alibaba,China,Industry,"Language,Multimodal,Vision,Audio",parameters,,,Likely,"""scales to hundreds of billions of parameters""",,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2604.15804,CC BY 4.0 (Epoch AI)
model,Qwen 3.6 Plus,2026-04-01,Alibaba,China,Industry,"Language,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwen.ai/blog?id=qwen3.6,CC BY 4.0 (Epoch AI)
model,MolmoAct 2,2026-04-05,"Allen Institute for AI,University of Washington,National University of Singapore,University of Pennsylvania,Johns Hopkins University,Amazon,Cortex AI",Singapore; United States of America,Mixed or other,"Vision,Language,Robotics",parameters,5485309424,5485309424,Likely,From https://huggingface.co/allenai/MolmoAct2/blob/main/model.safetensors.index.json ,,,,Honeybee to mouse,1.1e+22,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://arxiv.org/abs/2605.02881,CC BY 4.0 (Epoch AI)
model,Claude Mythos Preview,2026-04-07,Anthropic,United States of America,Industry,"Language,Multimodal",parameters,,,Unknown,Undisclosed; unconfirmed industry estimates claim ~8T total parameters. See https://finance.yahoo.com/technology/ai/articles/bytedance-targets-mega-ai-model-044310284.html,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://www-cdn.anthropic.com/7624816413e9b4d2e3ba620c5a5e091b98b190a5/Claude%20Mythos%20Preview%20System%20Card.pdf
https://www.anthropic.com/glasswing",CC BY 4.0 (Epoch AI)
model,GLM-5.1,2026-04-07,Z.ai (Zhipu AI),China,Industry,Language,parameters,754000000000,754000000000,Confident,Based on HF page: https://huggingface.co/zai-org/GLM-5.1,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://z.ai/blog/glm-5.1,CC BY 4.0 (Epoch AI)
model,Muse Spark,2026-04-08,Meta AI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.meta.com/blog/introducing-muse-spark-msl/,CC BY 4.0 (Epoch AI)
model,EXAONE 4.5,2026-04-09,LG AI Research,South Korea,Industry,"Language,Multimodal",parameters,33200000000,33200000000,Confident,"""Architecturally, EXAONE 4.5 integrates a 1.2B parameter vision encoder into the robust EXAONE 4.0 32B base model,""",,,,Honeybee to mouse,3.85e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"Eunbi Choi, Kibong Choi, Sehyun Chun, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Ahra Jo, Hyunjik Jo, Yeonsik Jo, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Changhun Lee, Haeju Lee, Jinsik Lee, Kyungmin Lee, Sangha Park, Kwangrok Ryoo, Minju Seo, Sejong Yang, Heuiyeen Yeen, Hwan Chang, Stanley Jungkyu Choi, Yejin Choi, Kyubeen Han, Joonwon Jang, Kijeong Jeon, Geunyeong Jeong, Gerrard Jeongwon Jo, Jiyeon Jung, Daeseong Kim, Dohoon Kim, Dohyun Kim, Hyunseo Kim, Minu Kim, Myoungshin Kim, Youchul Kim, Byungoh Ko, Christopher Lee, Edward Hwayoung Lee, Honglak Lee, Jiyoung Lee, Sangeun Lee, Seungwon Lim, Woohyung Lim, Jueun Mun, Jaewoo Park, Jimin Park, Jinho Park, Yongmin Park, Wooseok Seo, Yongwoo Song, Sihyuk Yi, Kyungjae Yoo, Sangyeon Yoon",CC BY 4.0 (Epoch AI)
model,GPT-5.4-Cyber,2026-04-14,OpenAI,United States of America,Industry,"Cybersecurity,Language",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/scaling-trusted-access-for-cyber-defense/,CC BY 4.0 (Epoch AI)
model,Gemini Flash 3.1 TTS,2026-04-15,Google DeepMind,United States of America,Industry,Audio,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-flash-tts/,CC BY 4.0 (Epoch AI)
model,Claude Opus 4.7,2026-04-16,Anthropic,United States of America,Industry,Language,parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-opus-4-7,CC BY 4.0 (Epoch AI)
model,Kimi K2.6,2026-04-20,Moonshot,China,Industry,Language,parameters,1040000000000,1040000000000,Confident,,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.kimi.com/ai-models/kimi-k2-6,CC BY 4.0 (Epoch AI)
model,GPT Image 2,2026-04-21,OpenAI,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-chatgpt-images-2-0/,CC BY 4.0 (Epoch AI)
model,GPT-5.5,2026-04-23,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-5/,CC BY 4.0 (Epoch AI)
model,GPT-5.5 Pro,2026-04-23,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Likely,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-5-5/,CC BY 4.0 (Epoch AI)
model,MiMo-V2.5-Pro,2026-04-23,Xiaomi Corp,China,Industry,Language,parameters,1020000000000,1020000000000,Likely,"""MiMo-V2.5-Pro is a 1.02T-parameter Mixture-of-Experts model with 42B active parameters, """,yes,,,Mouse to cat,6.804e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://mimo.xiaomi.com/mimo-v2-5-pro/,CC BY 4.0 (Epoch AI)
model,Tencent Hy3 preview,2026-04-23,Tencent,China,Industry,Language,parameters,295000000000,295000000000,Confident,"""Today, we are releasing and open-sourcing Hy3 preview. It is a 295B-parameter Mixture-of-Experts (MoE) model, with 21B active parameters, """,yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://hy.tencent.com/hy3-preview,CC BY 4.0 (Epoch AI)
model,DeepSeek-V4-Flash,2026-04-24,DeepSeek,China,Industry,Language,parameters,284000000000,284000000000,Likely,"284B total, 13B active",yes,,,Mouse to cat,2.496e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf,CC BY 4.0 (Epoch AI)
model,DeepSeek-V4-Pro,2026-04-24,DeepSeek,China,Industry,Language,parameters,1600000000000,1600000000000,Likely,"1.6T total, 49B active",yes,,,Mouse to cat,9.702e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf,CC BY 4.0 (Epoch AI)
model,GPT-5.5 Instant,2026-05-05,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/gpt-5-5-instant/,CC BY 4.0 (Epoch AI)
model,GPT-5.5 Cyber,2026-05-07,OpenAI,United States of America,Industry,Cybersecurity,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/gpt-5-5-with-trusted-access-for-cyber/,CC BY 4.0 (Epoch AI)
model,TML-Interaction-Small,2026-05-11,Thinking Machines,United States of America,Industry,"Language,Multimodal,Audio,Video",parameters,276000000000,276000000000,Confident,"""The current TML-Interaction-Small is a 276B parameter MoE with 12B active.""",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://thinkingmachines.ai/blog/interaction-models/,CC BY 4.0 (Epoch AI)
model,Composer 2.5,2026-05-18,Cursor,United States of America,Industry,Language,parameters,1040000000000,1040000000000,Likely,"Based on Kimi K2.5; 1.04T total, 32B active parameters",yes,,,Mouse to cat,3.87e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://cursor.com/blog/composer-2-5,CC BY 4.0 (Epoch AI)
model,Gemini 3.5 Flash,2026-05-19,Google DeepMind,United States of America,Industry,"Language,Multimodal,Audio,Video,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://deepmind.google/models/model-cards/gemini-3-5-flash/,CC BY 4.0 (Epoch AI)
model,Claude Opus 4.8,2026-05-28,Anthropic,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-opus-4-8,CC BY 4.0 (Epoch AI)
model,MiniMax-M3,2026-06-01,MiniMax,China,Industry,"Language,Multimodal,Vision,Video",parameters,428000000000,428000000000,Confident,"428B total, 22B active",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.minimax.io/blog/minimax-m3,CC BY 4.0 (Epoch AI)
model,MAI-Code-1-Flash,2026-06-02,Microsoft,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://microsoft.ai/news/introducingmai-code-1-flash/,CC BY 4.0 (Epoch AI)
model,MAI-Image-2.5,2026-06-02,Microsoft,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://microsoft.ai/news/introducing-mai-image-2-5/,CC BY 4.0 (Epoch AI)
model,MAI-Image-2.5-Flash,2026-06-02,Microsoft,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://microsoft.ai/news/introducing-mai-image-2-5/,CC BY 4.0 (Epoch AI)
model,Qwen3.7-Plus,2026-06-03,Alibaba,China,Industry,"Language,Vision,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://qwen.ai/blog?id=qwen3.7-plus

https://www.qwencloud.com/models/qwen3.7-plus",CC BY 4.0 (Epoch AI)
model,Nemotron 3 Ultra,2026-06-04,Nvidia,United States of America,Industry,Language,parameters,550000000000,550000000000,Likely,"""550B total parameters with up to 55B active per token"" from https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html ",yes,,,Mouse to cat,6.6e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2512.20856
https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf ",CC BY 4.0 (Epoch AI)
model,Claude Fable 5,2026-06-09,Anthropic,United States of America,Industry,"Language,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-fable-5-mythos-5,CC BY 4.0 (Epoch AI)
model,Claude Mythos 5,2026-06-09,Anthropic,United States of America,Industry,"Language,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-fable-5-mythos-5,CC BY 4.0 (Epoch AI)
model,Solar Open2 250B,2026-06-28,Upstage,South Korea,Industry,Language,parameters,250300000000,250300000000,Likely,"Total: 250.3B, Active: 14.7B per token. Mixture-of-Experts (320 routed experts + 1 shared expert).",yes,,,Mouse to cat,1.05e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/upstage/Solar-Open2-250B,CC BY 4.0 (Epoch AI)
model,LongCat-2.0,2026-06-29,Meituan Inc,China,Industry,Language,parameters,1600000000000,1600000000000,Likely,"1.6T total, 48B active",yes,,,Mouse to cat,1.008e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://longcat.chat/blog/longcat-2.0/,CC BY 4.0 (Epoch AI)
model,Tencent Hy3,2026-07-06,Tencent,China,Industry,Language,parameters,295000000000,295000000000,Confident,"""Hy3 is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters""",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/tencent/Hy3,CC BY 4.0 (Epoch AI)
model,Inkling,2026-07-15,Thinking Machines,United States of America,Industry,"Language,Multimodal,Audio,Video,Vision",parameters,975000000000,975000000000,Likely,"975B total, 41B active",yes,,,Mouse to cat,1.1e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://thinkingmachines.ai/news/introducing-inkling/,CC BY 4.0 (Epoch AI)
model,Kimi K3,2026-07-16,Moonshot,China,Industry,"Language,Multimodal,Vision",parameters,2800000000000,2800000000000,Speculative,104 billion active parameters,yes,yes,,Cat or more,2.0001e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/moonshotai/Kimi-K3,CC BY 4.0 (Epoch AI)
model,Qwen 3.8 Max,2026-07-19,Alibaba,China,Industry,"Language,Vision,Video",parameters,2400000000000,2400000000000,Confident,"2.4T total, 95B active",yes,,,Cat or more,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://qwen.ai/blog?id=qwen3.8,CC BY 4.0 (Epoch AI)
model,Claude Opus 5,2026-07-24,Anthropic,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/news/claude-opus-5,CC BY 4.0 (Epoch AI)
model,A.X K2,2026-07-29,SK Telecom,South Korea,Industry,Language,parameters,688000000000,688000000000,Confident,"MoE architecture, total parameters 688B, active parameters 33B (61 layers: 1 dense + 60 MoE; 256 routed experts + 1 shared, 8 active per token)",yes,,,Mouse to cat,1.8e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/skt/A.X-K2
https://github.com/SKT-AI/A.X-K2/blob/main/A_X_K2_Tech_Report.pdf",CC BY 4.0 (Epoch AI)
model,K-EXAONE 2.0,2026-07-31,LG AI Research,South Korea,Industry,Language,parameters,750000000000,750000000000,Confident,"MoE architecture: 750B total parameters, ~37B activated per token. 78 layers (2 dense + 76 MoE); 256 routed experts + 1 shared expert, top-8 routed experts activated per token. Upcycled from K-EXAONE (236B total / 23B active) by depth expansion (48 -> 78 layers) and expert duplication (128 -> 256 ex",yes,,,Mouse to cat,3.55e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B
https://github.com/LG-AI-EXAONE/K-EXAONE-2.0
https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B/blob/main/assets/K-EXAONE-2.0-Technical-Report.pdf",CC BY 4.0 (Epoch AI)
model,Muse Spark 1.2,2026-08-05,Meta AI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.meta.ai/blog/introducing-muse-code-and-muse-spark-1-2,CC BY 4.0 (Epoch AI)
model,SeedRealtime,2026-08-05,ByteDance,China,Industry,"Language,Multimodal,Video,Audio",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://seed.bytedance.com/en/blog/seedrealtime-audio-visual-full-duplex-llm-released-toward-omni-modal-natural-interaction 

https://seed.bytedance.com/en/SeedRealtime",CC BY 4.0 (Epoch AI)
model,Motif-3,2026-08-07,Motif Technologies,South Korea,Industry,Language,parameters,314000000000,314000000000,Confident,"Fine-grained sparse MoE: approximately 314B total parameters, ~13.2B activated per token. 53 Transformer layers (2 dense + 51 MoE); each MoE layer has 384 routed experts + 1 shared expert, with top-8 routing. Grouped Differential Latent Attention (GDLA), modified manifold-constrained hyper-connectio",yes,,,Mouse to cat,1.1e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/Motif-Technologies/Motif-3
https://huggingface.co/Motif-Technologies/Motif-3/blob/main/motif_3_technical_report.pdf
https://github.com/MotifTechnologies/motif3-training-example",CC BY 4.0 (Epoch AI)
model,Qwen3.8-2.4T-A95B,2026-08-08,Alibaba,China,Industry,Language,parameters,2400000000000,2400000000000,Confident,"2.4T total, 95B active",yes,,,Cat or more,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B,CC BY 4.0 (Epoch AI)
model,GPT-5.6 Cyber,2026-08-11,OpenAI,United States of America,Industry,"Cybersecurity,Language",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/expanding-daybreak-as-the-cyber-defense-window-narrows/,CC BY 4.0 (Epoch AI)
model,GLM-5.3,2026-08-14,Z.ai (Zhipu AI),China,Industry,Language,parameters,744000000000,744000000000,Confident,"""Same base model as GLM-5.2"", which had 744B total and 40B active.",yes,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://z.ai/blog/glm-5.3,CC BY 4.0 (Epoch AI)
model,GEN-1.5,2026-08-19,Generalist,United States of America,Industry,"Robotics,Language,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://generalistai.com/blog/gen-1.5,CC BY 4.0 (Epoch AI)
model,GLM-5.3-Flash,2026-08-20,Z.ai (Zhipu AI),China,Industry,"Language,Vision,Multimodal",parameters,320000000000,320000000000,Likely,"320B total, 18B active.",yes,,,Mouse to cat,3.24e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://z.ai/blog/glm-5.3-flash,CC BY 4.0 (Epoch AI)
model,Tencent Hy4 preview,2026-08-28,Tencent,China,Industry,Language,parameters,770000000000,770000000000,Likely,,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://hy.tencent.com/hy4-preview,CC BY 4.0 (Epoch AI)
model,Claude Fable 5.1,2026-09-01,Anthropic,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/claude-fable-and-mythos-5-1,CC BY 4.0 (Epoch AI)
model,Gemini 3.8 Flash,2026-09-02,Google DeepMind,United States of America,Industry,"Language,Multimodal,Image generation,Video,Audio",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://ai.google.dev/gemini-api/docs/models/gemini-3.8-flash,CC BY 4.0 (Epoch AI)
model,Muse Spark 1.3,2026-09-02,Meta AI,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.meta.ai/blog/introducing-muse-spark-1-3,CC BY 4.0 (Epoch AI)
model,GPT-6 Astra,2026-09-03,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Likely,,,,,,9.4e+26,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,,CC BY 4.0 (Epoch AI)
model,OpenEvidence Darwin,2026-09-03,OpenEvidence,,Not stated,"Language,Medicine",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.openevidence.com/blog/model-family,CC BY 4.0 (Epoch AI)
model,WeatherNext 3,2026-09-03,"Google DeepMind,Google Research",United States of America,Industry,Earth science,parameters,,,Speculative,"""Generating a single ensemble member forecast takes roughly 6.3 minutes of wall clock time using 4 TPUv5p chips.""; can infer that it fits within 4x95GB (memory/TPUv5p) = 380GB of memory; <380B at FP8, <190B at FP16, <95B at FP32.",,,,,2.0725198384e+23,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://arxiv.org/abs/2609.03582 

https://blog.google/innovation-and-ai/models-and-research/google-deepmind/introducing-weathernext-3/ ",CC BY 4.0 (Epoch AI)
model,ChatGPT Images 2.5,2026-09-08,OpenAI,United States of America,Industry,"Image generation,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-chatgpt-images-2-5/,CC BY 4.0 (Epoch AI)
model,GPT-Image-2.5 Flare,2026-09-08,OpenAI,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-chatgpt-images-2-5/,CC BY 4.0 (Epoch AI)
model,GPT-Image-2.5 Sunburst,2026-09-08,OpenAI,United States of America,Industry,Image generation,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-chatgpt-images-2-5/,CC BY 4.0 (Epoch AI)
model,Mercury 2.5,2026-09-08,Inception Labs,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.inceptionlabs.ai/blog/introducing-mercury-2-5,CC BY 4.0 (Epoch AI)
model,DeepSeek V4.1 Flash,2026-09-09,DeepSeek,China,Industry,"Language,Vision,Multimodal",parameters,552000000000,552000000000,Likely,"552B total, (not counting 196B Engram parameters) 16B active during decode, 8B during prefill.",yes,,,Mouse to cat,4.32e+24,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf 

https://www.deepseek.com/en/news/deepseek-v4-1-flash/",CC BY 4.0 (Epoch AI)
model,Step 5 Preview,2026-09-18,StepFun,China,Industry,"Language,Vision,Multimodal",parameters,600000000000,600000000000,Likely,,,,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://platform.stepfun.ai/docs/en/guides/models/step-5-preview,CC BY 4.0 (Epoch AI)
model,Grok 4.7,2026-09-21,xAI,United States of America,Industry,Language,parameters,2100000000000,2100000000000,Likely,"Not confirmed in official release but Elon said ""Grok 4.7 will be the 2.1T model released a few weeks later. This will be better than 4.6 in every way, except slightly slower to serve, albeit with even better token efficiency."" on July 28, 2026.  On Sepetember 14 he said ""It’s unequivocally better t",,yes,,Mouse to cat,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/news/grok-4-7,CC BY 4.0 (Epoch AI)
model,Claude Opus 5.5,2026-09-22,Anthropic,United States of America,Industry,Language,parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/claude-opus-5-5,CC BY 4.0 (Epoch AI)
model,GPT-6 Luna,2026-09-22,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-6-sol-and-luna/,CC BY 4.0 (Epoch AI)
model,GPT-6 Luna Decisions,2026-09-22,OpenAI,United States of America,Industry,"Language,Vision,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/decisions-api/,CC BY 4.0 (Epoch AI)
model,GPT-6 Sol,2026-09-22,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-6-sol-and-luna/,CC BY 4.0 (Epoch AI)
model,Claude Sonnet 5.5,2026-09-28,Anthropic,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://www.anthropic.com/claude-sonnet-5-5,CC BY 4.0 (Epoch AI)
model,GPT-6.1 Sol,2026-09-29,OpenAI,United States of America,Industry,"Multimodal,Language,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://openai.com/index/introducing-gpt-6-1-sol/,CC BY 4.0 (Epoch AI)
model,Gemini 4 Argon,2026-09-30,Google DeepMind,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/,CC BY 4.0 (Epoch AI)
model,Mistral Large 4,2026-10-06,Mistral AI,France,Industry,"Language,Multimodal,Vision",parameters,1050000000000,1050000000000,Confident,"1.05T total, 49B active; 52B including embedding and output layers. Includes 1.6B vision encoder.",yes,,,Mouse to cat,1.24e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://mistral.ai/news/mistral-large-4/

https://mistral.ai/news/mistral-large-4-oss/ 

https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/blob/main/assets/technical_report.pdf ",CC BY 4.0 (Epoch AI)
model,Muse Wonder,2026-10-12,Meta AI,United States of America,Industry,"Language,Vision,Video,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://research.meta.ai/blog/muse-wonder,CC BY 4.0 (Epoch AI)
model,Muse Wonder Full,2026-10-12,Meta AI,United States of America,Industry,"Language,Vision,Video,Multimodal",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,"https://research.meta.ai/blog/muse-wonder

https://research.meta.ai/static/muse-wonder-model-card",CC BY 4.0 (Epoch AI)
model,Grok Voice 3.0,2026-10-14,xAI,United States of America,Industry,"Language,Audio,Speech",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://x.ai/news/grok-voice-3,CC BY 4.0 (Epoch AI)
model,MAI-Base-2,2026-10-15,Microsoft,United States of America,Industry,"Language,Vision,Multimodal",parameters,2800000000000,2800000000000,Confident,"""80 billion active and 2.8 trillion total parameters.""",yes,,,Cat or more,2.4e+25,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://microsoft.ai/pdf/Frontier-Pre-training-at-Microsoft-AI.pdf?v=2,CC BY 4.0 (Epoch AI)
model,Gemini 4 Carbon,2026-10-28,Google DeepMind,United States of America,Industry,"Language,Multimodal,Vision",parameters,,,Unknown,,,,,,,,,published or estimated by Epoch AI (see epoch_confidence),,,epoch,https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/,CC BY 4.0 (Epoch AI)
animal,Roundworm (C. elegans),,,,,Worm,synapses,5000,8000,,"Low: White et al. 1986, a typical animal, whole nervous system. High: Witvliet et al. 2021, adults, brain (nerve ring and ventral ganglion). Chemical synapses only; the 2,000 neuromuscular junctions and 600 gap junctions are not counted.",,,,,,302,"whole nervous system, hermaphrodite",counted,Cognitron (1975-09-01),Cognitron (1975-09-01),white1986;witvliet2021,,see source ids
animal,Sea squirt larva (Ciona),,,,,Sea squirt,synapses,4846,6618,,"High: all 6,618 synapses as printed. Low: IQ Metrics subtraction of the 1,772 neuromuscular junctions. Gap junctions (1,206) not counted.",,,,,,177,"whole central nervous system, tadpole larva",counted,Cognitron (1975-09-01),Cognitron (1975-09-01),ryan2016,,see source ids
animal,"Marine worm larva (Platynereis, 3 days)",,,,,Marine worm,synapses,26881,32381,,"Low: the 26,881 synapses inside the published connectome graph. High: the 32,381 synapses identified before graph filtering.",,,,,,,"whole body, 3-day larva",counted,Neocognitron (1980-04-01),Neocognitron (1980-04-01),verasztoe2025,,see source ids
animal,Fruit fly larva (first instar),,,,,Fruit fly,synapses,544000,544000,,One count (Winding et al. 2023).,,,,,,3013,"whole brain, first-instar larva",counted,Neocognitron (1980-04-01),Neocognitron (1980-04-01),winding2023,,see source ids
animal,Zebrafish larva (6 days),,,,,Zebrafish,synapses,25250000,25250000,,"Preprint, automatic detection, not proofread. The abstract says more than 25 million synapses and more than 240,000 cells; the text says 176,810 cells and 25.25 million synapses; we use the text.",,,,,,,"whole brain, retina and anterior spinal cord, algorithmic reconstruction",counted,Support Vector Machines (1995-09-01),Support Vector Machines (1995-09-01),xie2025,,see source ids
animal,Fruit fly (adult),,,,,Fruit fly,synapses,46000000,312000000,,"Three printed values for the adult fly. Low: 46 million presynaptic sites (Berg et al. 2026, male brain and nerve cord, preprint text). Middle: 54.5 million synapses (Dorkenwald et al. 2024, female whole brain). High: 312 million postsynaptic densities connected to those presynaptic sites (Berg et al.). The range is two ways of counting one animal's wiring.",,,,,,139255,"whole brain (FlyWire, female); male central nervous system (Berg et al.)",counted,Support Vector Machines (1995-09-01),KN-LM (2007-06-22),dorkenwald2024;berg2026,,see source ids
animal,Honeybee,,,,,Honeybee,synapses,391370000,8200000000,,"We found no paper that prints a synapse count for a honeybee brain. Bracket = one million neurons times the synapses per neuron measured in the fruit fly brain (low) and in the mouse neocortex (high). An IQ Metrics calculation, not a count.",,,,,,1000000,"whole brain, estimate (no count published)",calculated,KN-LM (2007-06-22),KN-LM (2007-06-22),vetsci2025;ctr2026;dorkenwald2024;schuz1989,,see source ids
animal,Mouse (neocortex),,,,,Mouse,synapses,81000000000,81000000000,,"Density x volume (Schuz and Palm 1989: 7.2 x 10^8 synapses per mm3 x 112 mm3). Neocortex only, so a lower bound for the whole mouse brain, which no paper counts.",,,,,,10000000,"neocortex, both hemispheres",counted,SB-LM (2007-06-22),SB-LM (2007-06-22),schuz1989,,see source ids
animal,Cat (neocortex),,,,,Cat,synapses,2239000000000,2439000000000,,"IQ Metrics calculation. Synapses per neuron from the cat motor cortex (area 4 gamma): (216 + 43) million per mm3 / 28,900 neurons per mm3 = 8,962 (enriched) and (247 + 41) / 29,500 = 9,763 (impoverished). Applied to the cortical neuron count of one cat. One cortical area stands for the whole cortex.",,,,,,249830000,"cerebral cortex, calculated from neurons and synapses per neuron",calculated,Grok 3 (2025-02-17),Grok 3 (2025-02-17),jardim2017;beaulieu1989,,see source ids
animal,Macaque (neocortex),,,,,Macaque,synapses,3123744000000,6474000000000,,"IQ Metrics calculation. 3,900 synapses per neuron was measured in the primary visual cortex, where neurons are packed densest, so the whole-cortex figure is probably a lower bound. The two ends are two macaque species with one animal each.",,,,,,800960000,"cerebral cortex, calculated from neurons and synapses per neuron",calculated,none yet,none yet,gabi2010;beaulieu1992,,see source ids
animal,Human (neocortex),,,,,Human,synapses,150000000000000,164000000000000,,"The 2003 figure appears in a review of neurostereology studies, and we found no independent second count, so the two are one estimate. Neocortex only: cerebellum, thalamus and the rest of the brain are not counted, so the whole brain holds more. Round figures such as 100 trillion appear in textbooks; we found no counting study behind them.",,,,,,16300000000,"neocortex, both hemispheres",counted,none yet,none yet,pakkenberg2003;tang2001;azevedo2009,,see source ids
sample,"Mouse, 1 cubic millimetre of visual cortex (MICrONS)",,,,,,synapses,524000000,524000000,,"524 million synaptic clefts across two sub-volumes; 82,247 neurons detected in the larger one",,,,,,,"1 cubic millimetre of cortex, not a whole brain","counted (automatic, not all proofread)",,,microns2025,,see source ids
sample,"Human, 1 cubic millimetre of temporal cortex (H01)",,,,,,synapses,133700000,133700000,,"133.7 million synaptic connections; 57,216 cells, about 16,000 of them neurons",,,,,,,"1 cubic millimetre of cortex, not a whole brain",counted (automatic),,,shapsoncoe2024,,see source ids