Tech Times

Hermes MoA 2.0 Combines GPT, Claude and DeepSeek to Outscore Any One Model

Nous Research has released Hermes Mixture of Agents 2.0 (MoA 2.0), an update to its open-source Hermes Agent framework that ...
Tech Times

LLM Data Mixture Breaks When Training Pools Shift: Causal Inference Offers Fix

LLM training data mixture optimization breaks when training pools shift — every prior proxy experiment becomes stale.
IEEE

GMDR-Net: A Lightweight OBB-Based SAR Ship Detection Model Based on Gaussian Mixture Data Augmentation and Distance Rotation IOU Loss

Abstract: Aiming at the problems of oriented-bounding-box-based (OBB-based) synthetic aperture radar (SAR) ship detection, which include the large model volume, the imbalance of positive and negative ...
GitHub

TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts

We revised TESTAM routing and fixed some issues -- e.g., some of the previous pseudo label-generation process are problematic with deprecated codes We partially updated TESTAM to be usable for the ...
GitHub

MoH: Multi-Head Attention as Mixture-of-Head Attention

We propose Mixture-of-Head attention (MoH), a new architecture that treats attention heads as experts in the Mixture-of-Experts (MoE) mechanism. MoH has two significant advantages: First, MoH enables ...
IEEE

MC#: Mixture Compressor for Mixture-of-Experts Large Models

Abstract: Mixture-of-Experts (MoE) has emerged as an effective and efficient scaling mechanism for large language models (LLMs) and vision-language models (VLMs). By expanding a single feed-forward ...