Toasted Almond Party Spread

1 (8 ounce) package cream cheese, softened
1 1/2 cups shredded Swiss cheese
1/2 cup sliced almonds, toasted, divided
1/3 cup mayonnaise
2 tablespoons sliced green onions
1/8 teaspoon pepper
1/8 teaspoon ground nutmeg
Assorted crackers

In a small mixing bowl, beat the cream cheese until smooth. Stir in the Swiss cheese, 1/3 cup almonds, mayonnaise, onions, pepper and nutmeg. Spoon onto a lightly greased pie plate. Bake at 350 degrees F for 14-15 minutes or until heated through. Sprinkle with remaining almonds. Serve warm with crackers.